The Ultimate Guide to Migrating from a Monolithic Automation Platform
Whitepaper
October 2024
19 min read
Introduction
Processes are the algorithms that determine how an organization runs. They define how teams work together, how the organization works with partners and suppliers, and how it delivers value to its customers. To deliver amazing customer experiences, keep up with competitors, streamline operations, and improve their bottom line, organizations must automate the processes that are core to their business. And it doesn’t stop at automating processes; as AI and automation rapidly advance, organizations must also future-proof their enterprise architecture to remain agile and competitive.
To meet the urgent need for process automation, many organizations purchase a monolithic automation platform. These products are positioned as all-in-one solutions for automating business processes that involve both automated tasks and tasks that must be executed by knowledge workers.
What types of monolithic automation platforms are available?
Traditional monolithic automation platforms originated in the world of business process management suites (known as BPMS or iBPMS). These products originally addressed the need to automate stable, repeatable back-office processes for departments such as HR and Finance. As digital transformation has driven the demand to automate more of the business, vendors expanded these products to include adjacent automation capabilities.
Modern monolithic automation platforms have emerged as vendors add lightweight process automation capabilities to products that were originally designed for purposes such as customer relationship management (CRM), IT service management (ITSM), and personal and team productivity. In some cases, these capabilities were built in-house; in other cases, process-related features have been loosely integrated with the original product after various acquisitions.
What makes a platform an automation monolith?
The defining quality of monolithic automation platforms is their architecture. A monolith is architected as a single, black-box application as opposed to a collection of loosely coupled components or microservices that can operate independently.
The monolithic automation platforms that are on the market today have a few other things in common. First, they come with a built-in proprietary design environment where users build processes, business rules, and basic user interfaces. While business technologists and citizen developers may find built-in design tooling useful at first, proprietary tools often hide the complexity of real-world business processes, making them difficult to visualize and control.
Second, automation monoliths have limited options for implementing complex or non-standard processes that require custom code. The professional software developers who would write this code must either learn how to use the monolith’s proprietary tooling or use the specific programming language that the monolith requires. These requirements hinder developers’ productivity and, if there’s a skills gap, may require you to hire expensive consultants.
Third, the closed nature of automation monoliths often includes their approach to integrating with and orchestrating other enterprise tools and technologies. When connectors for enterprise tools can’t be modified or extended for your specific use cases, it slows down process automation projects and greatly increases the risk of vendor lock-in.
The challenges of monolithic automation platforms
Organizations that adopt monolithic automation platforms to address process automation and orchestration use cases encounter several challenges.
First, while they’re under pressure to drive return on investment (ROI) and realize value, monolithic automation platforms tend to slow down value delivery because they struggle to deal with complex or non-standard processes. When extensive customization is required to automate business-critical processes, projects stall, and ROI decreases. Customization also leads to vendor lock-in and technical debt, both of which further impact ROI.
Second, automation monoliths don’t provide sufficient transparency or control for business processes, especially complex or highly specialized processes that cross multiple business units. Without a complete view of a process from end to end, opportunities to improve process performance are limited.
Third, automation monoliths don’t enable you to adapt to changing customer needs, market pressures, or regulatory requirements. Inflexible process implementation requirements and cumbersome customizations can mean it takes weeks or even months to roll out process changes. This negatively impacts customer experience and hinders innovation; it can even mean that you miss out on opportunities to win over competitors.
An alternative to monolithic automation platforms
Future-proofing your enterprise architecture requires automation solutions that can be adapted to your organization’s unique needs. Camunda’s modular, flexible, and open approach offers rapid implementation, faster time-to-value, and faster realization of value compared to monolithic automation platforms. Camunda empowers you to rapidly adapt to evolving market forces and emerging technologies, providing a platform that enables both IT and business teams to respond quickly.
With Camunda, you maintain full control and transparency over every aspect of your process automation, unlike monolithic automation platforms that require teams to use proprietary tools and limit customization.
This guide provides practical examples of companies that have successfully migrated from a monolithic automation platform to Camunda’s process automation and orchestration solution.
Key considerations for replacing a monolithic automation platform
These are key considerations to keep in mind when you’re considering migrating away from a monolithic automation platform to a composable process automation and orchestration solution such as Camunda.
Support for processes that combine automated and manual work
When replacing a monolithic automation platform, consider the volume and variety of systems and services that are required to execute a business process from start to finish. Without a solution that can orchestrate a process from end to end, processes can end up fragmented into different parts that are executed in isolation. This fragmentation causes a lack of visibility, integration, and control of the end-to-end process, slowing down or even preventing effective troubleshooting, reporting, and analysis. Look for a new solution that orchestrates both automated tasks and work that must be done by people, no matter how many applications or systems are involved.
Support for long-running processes
Many organizations have business processes that can run for hours, days, weeks, or even longer. Long-running processes present a variety of technical challenges, such as tracking the state of a process, correlating all activities and data related to the process, and triggering timeouts. Also, long-running processes often lead to additional business requirements; for example, if the payment for an online order fails, the customer might be allowed a certain number of days to retry with a different payment method.
Camunda process orchestration is based on the ISO-standard Business Process Model and Notation (BPMN), which allows teams to design processes that are both graphical and executable. BPMN has built-in support for long-running processes; it automatically handles technical challenges such as state persistence, data correlation, and timeouts. BPMN also facilitates monitoring, alerting, and reporting on the process level, which are key features for managing long-running processes.
Deep process analytics and optimization
Most, if not all, monolithic automation platforms provide reporting about the data they collect, often including dashboards with graphs that visualize data. However, these reports may lack the context of the end-to-end business process that invoked tasks that were executed by the monolith itself, which means teams have an incomplete picture of process performance. This incomplete picture makes it difficult or impossible to identify bottlenecks and other areas where the organization can take action to improve processes.
With a solution such as Camunda that orchestrates processes across all people, systems, and devices, you have a 360-degree view of process execution data. This data results in a deep set of analytics, complete with intuitive visualizations and heatmaps that are useful for both technical and business stakeholders.
Consistent process model visualization
Many automation monoliths do not provide users with a consistent process model visualization across design, monitoring, and improvement activities. When the process model is only visible at design time, collaboration between business users and IT is limited; it makes it much harder or even impossible for business users to monitor running processes and report on process performance. It also complicates cross-organization information sharing and reporting to regulatory organizations.
In contrast, Camunda takes a “one model” approach. At design time, business and IT users use BPMN to build visual representations of complex business processes that are executed directly by Camunda’s workflow engine, Zeebe. When monitoring running processes, users see information about process status and incidents overlaid on the same model, so they don’t have to interpret technical performance data or decipher server logs. Reports show historical process execution data with the same visualization, including heatmaps that provide an intuitive way to understand process performance.
Collaboration between IT and the business
Bridging the gap between IT and the business is a challenge for every organization, especially with process automation playing a key role in digital transformation. Business and IT stakeholders often have different goals, incentives, and priorities, and these differences tend to slow down communication, prevent alignment on project priorities, and cause implementation errors.
When replacing a monolithic automation platform, look for a solution such as Camunda that uses the globally recognized standards of Business Process Model and Notation (BPMN) and Decision Model and Notation (DMN). Standards are a common language that all stakeholders can speak, so nothing is lost in translation between business requirements and technical implementation. BPMN and DMN allow business users to create visual process diagrams and decision tables, while also allowing technical users to round out the technical implementation of automated processes and decisions by editing the underlying code.
Developer friendliness for complex processes
Monolithic automation platforms take a vendor-specific approach to application development with the goal of minimizing the amount of code that needs to be written. However, core business processes are complex and require bespoke solutions, so organizations run into trouble as soon as they need to implement requirements that are outside the realm of what their automation tool supports. To work around technical and functional limitations, software developers have to learn the vendor-specific way of automating tasks, which takes time and can lead to implementations that aren’t optimized for performance, maintenance, or stability.
Camunda is designed with developers in mind, so they can quickly start automating processes without learning a vendor-specific development framework or being forced to use a proprietary IDE. Teams can use the type of environment they’re familiar with when creating, testing, and operating the applications they develop. For example, they can work in their preferred code editor, program in the language they like, store their code in a version control system, automatically test their code, implement continuous integration, and manage their containerized applications on a platform such as Kubernetes. There’s no need to adjust to a Camunda-specific way of working.
Low total cost of ownership
Many factors contribute to a product’s total cost of ownership: licensing, training courses, the initial development period before moving to production, the turnaround time for changes and new developments, hiring consultants for projects that internal teams can’t complete, the cost of the infrastructure needed to run the product, and so on. Most monolithic automation platforms have a reputation for high total cost of ownership due to long ramp-up times (sometimes measured in years), ongoing consulting fees, and high infrastructure costs.
Camunda approaches total cost of ownership from several angles:
- Camunda’s developer-friendly approach and use of open standards makes it easy to get started and reduces time-to-value because the IT team doesn’t have to spend time learning a vendor-specific development framework or proprietary development tools.
- Camunda’s open architecture allows teams to choose which components to use, so they can easily integrate Camunda into the organization’s existing tech stack.
- Camunda is a lightweight solution that requires few infrastructure resources, and that can run on premises or in a public, private, or hybrid cloud.
Scalability and resilience
As you consider migrating away from a monolithic automation platform, also consider the way your business processes need to scale in the future. Camunda’s next-generation workflow engine, Zeebe, is designed to accommodate high-throughput use cases out of the box. While traditional workflow engines rely on a central database that causes a performance bottleneck, Zeebe uses event-streaming technology instead, enabling it to deliver unparalleled scalability and performance. It maintains the state of running process instances in a way that scales up with high transaction volumes, that’s resilient to failures, and that performs well at scale.
In addition, Zeebe has a distributed architecture that ensures resilience, even when load is high. This distributed architecture is ideal for geo-replication that further protects data. Zeebe distributes data across all brokers in a cluster with storage directly on the server filesystem, so if one broker goes down—or if a whole datacenter goes down—another can replace it with no data loss.
A Buyer’s Guide to End-to-End Process Orchestration
Monolithic automation platforms create silos that can lead to broken or inefficient end-to-end business processes and prevent you from reaching your strategic business goals. This guide shows how to determine whether your automation tech stack is holding you back and how end-to-end process orchestration can help.
Preparing to migrate from a monolithic automation platform
Once you’ve decided to migrate away from a monolithic automation platform in favor of another solution, it’s important to prepare for the cultural and technical changes that are about to take place. There are typically many different stakeholders involved in process automation, from software developers and enterprise architects to line-of-business leaders and business technologists. As a result, this type of project requires IT and business stakeholder alignment.
In addition, process automation initiatives are often part of a larger technology modernization effort. Depending on the size of the company, the migration process might be handled by a few IT staff members or by an automation center of excellence (CoE) team. Either way, it’s critical for a focused team to drive the migration process, ideally by starting with a proof of concept (PoC). The PoC should help the team:
- Verify that a specific approach or tool works within the use case
- Work through granular questions before kicking off a larger automation effort
- Showcase an example that gets internal stakeholders to buy into the larger automation effort
From there, the team driving the migration process should:
- Equip all stakeholders, especially the IT team, to use the solution
- Help line-of-business leaders understand that processes might look different, including user interfaces and process flows
- Enable all relevant teams to view processes and request changes if needed
How to migrate from a monolithic automation platform to Camunda
The technical aspects of migration from a monolithic automation platform to Camunda are summarized below. Camunda uses the BPMN standard for process modeling and the DMN standard for decision modeling. If you’re migrating from a platform that leverages BPMN and DMN, your journey will be easier than if the platform has a proprietary approach to modeling. Even so, these guidelines should serve as a solid foundation for planning the transition.
Step 1: Migrate or rebuild business process models
The first step in replacing a monolithic automation platform is to decide whether to migrate or rebuild your existing business process models. We recommend rebuilding process models instead of relying on an automated conversion; in our experience, it often takes less time to rebuild models than to create and thoroughly test an automated conversion tool. Rebuilding models also gives you the opportunity to improve processes that haven’t been revised in years and to leverage the full power of the BPMN process modeling standard. Rebuilding one process at a time during a PoC will give you an idea of how much time and effort it takes to migrate process models in your environment.
In some cases, it may make sense to use tools created by the Camunda consulting team to migrate process models from monolithic automation platforms. You can find these tools on GitHub. Keep in mind that these tools are assistants; none of them is a “silver bullet” that will transform automation from one tool to another with a single click. Instead, they will generate a BPMN file that attempts to retain the fidelity of the original process model. Depending on the tool you’re migrating from, you’ll still need to add code and Camunda Connectors to make the process executable.
Step 2: Migrate running process instances
If you have long-running business processes, you may find that migrating running process instances is unavoidable. There are two approaches to migrating process instances.
Parallel operation approach
With the parallel operation approach, you continue to operate the automation monolith until no process instances are left there. Meanwhile, you start new process instances on Camunda. To do so, you must implement switching logic to route traffic from incoming requests to the old or new system. Keep in mind that you’ll need to perform operational tasks such as checking failures, calculating KPIs, and performing instance counts on both systems.
The advantage of this approach is that, because you don’t need to migrate running process instances, you’ll save effort on this step. Also, any risk associated with going live with Camunda is reduced because the old system is still in place.
The disadvantage of this approach is that operating two systems in parallel for a long time requires effort. The switching logic is normally not easy to implement and test for all corner cases. Also, the team’s notification could be at risk, if people are eager to discard the old tooling and move on.
Big bang approach
With the big bang migration approach, you stop the automation monolith and ensure that all process instances have reached a wait state. Then, you migrate all instances to Camunda. This approach requires mapping all possible wait states from old process models to BPMN process definitions, reading the data from the old system, and running a migration script to create new process instances in Camunda in the correct state. After that, you start the new system.
The advantage of this approach is that after the migration, you only have to operate one system, and you can test the migration beforehand to avoid any surprises. Because of this, the big bang approach is often preferred over the parallel operation approach.
The disadvantage is that implementing and testing the migration script requires development effort. Also, the run time of migration scripts and additional health checks afterwards often require system downtime. In order to create process instances in the correct state, you do not want to trigger any service tasks or wait for human interaction along the way. Also, the desired state might exist in a hierarchy of subprocesses. These factors can make the migration project more complex and potentially require you to add elements to your BPMN process models solely to accommodate migration (if the Camunda API alone isn’t sufficient).
Case Study: Securing stakeholder buy-in for modernization
For Indiana Farm Bureau Insurance, stakeholder buy-in was a critical first step in their migration from an automation monolith to a composable solution built on Camunda. The development team had worked extensively with the previous platform and originally had a “if it’s not broken, don’t fix it” mentality. Additionally, experienced senior developers wanted to be sure that their work would be valued and that they’d be delivering something that’s beneficial for the business end user.
Jarvis Ka, enterprise architect at Indiana Farm Bureau Insurance, stressed the importance of setting expectations with the development team on the time required to get an end-to-end process orchestration solution into production. This involves adjusting expectations at both the scrum master and project manager levels. Many manager-level stakeholders may be resistant to change, so it’s important to explain how existing processes will transition to the new system. For example, a line of business manager might be concerned about how the user experience (UX) might change for the end user when filing a mobile insurance claim.
To cut down on resistance and demonstrate these changes, Ka and his team shared BPMN process models that were created during the PoC phase and annotated them with information describing each task in the process. From there, they created PDFs and sent them to each business stakeholder. This asynchronous communication reduced the number of cross-team meetings required by giving all stakeholders a visual representation of processes in the new solution.
Case Study: Transitioning from a monolith to microservices
Deutsche Telekom had a monolithic process automation system that created a number of issues affecting the business and the user experience:
- A lengthy time-to-market lasting more than 12 months
- Vendor lock-in limited the implementation of new features – it took five days to set up or make changes to environments
- Releases took an average of 1000 people-days, or roughly three months, to realize
- Regression testing took around two days to process all test cases
“We need to be on time. We can’t wait seven days to process an order or respond to something; we need to be always ahead and responsible,” said Willm Tüting, managing director of conology GmbH, who has worked with Deutsche Telekom IT for over 15 years.
Laying the groundwork for change
Deutsche Telekom IT took the first steps towards modernizing its processes by adopting a partially agile development approach, working in three-month sprints. “We created several processes to deliver small fixes faster, but it made life much more complex,” Tüting said.
All fixes had to be delivered together with larger change requests, which took considerable people-hours to accomplish. In addition, the organization was still struggling with the monolithic automation system, which didn’t allow for true agility.
At the same time, Deutsche Telekom began investing in fiber optic cables. With this significant upgrade in hardware came the opportunity to revolutionize Deutsche Telekom IT’s outdated systems. The result was a complete change in both the operating system and DevOps approach, guided by three goals:
- Speed up: Change development from Business Process Execution Language (BPEL) to Java and teams’ way of working from waterfall to agile
- Develop cross-functional teams: Change from skill-based teams to international cross-functional teams
- Increase efficiency: Increase development efficiency by using Camunda, Spring Boot, and other state-of-the-art technologies
From a monolith to microservices in the cloud
With these three goals in mind, Deutsche Telekom IT implemented a microservices-based architecture in the cloud with Camunda running within many services. According to Friedbert Samland, project manager for Deutsche Telekom IT, the new system is comprised of:
- Microservices: This approach partitioned the monolith and allows for cross-functional work. There is no graphical user interface; instead, it is now a pure BPMN system. “Inspired by Camunda co-founder Bernd Ruecker’s microservices workflow automation cheat sheet, we run Camunda engines in many microservices, talking directly to a message broker,” Friedbert said.
- Cloud: The power of the cloud dramatically reduces runtimes and enables a staged, fine-grained delivery approach. With no ready-made solution, Deutsche Telekom IT designed its own Kubernetes-based approach.
- Scaled Agile Framework (SAFe): Introducing a new organizational framework with shorter end-to-end cycle times has enabled flexibility and speed.
- DevOps philosophy: Automation and self service ensure quality and speed and are key to Deutsche Telekom IT’s DevOps philosophy.
Achieving process automation alignment
One of the greatest advantages of Deutsche Telekom IT’s Camunda revolution has been enabling compliance by default. As a globally distributed business with teams working across the world with multiple vendors and sensitive data, a highly automated solution was required. The resulting architecture now enables compliance by default and ensures data security.
Deutsche Telekom IT has also built its own internal process monitoring platform, inspired by the token concept employed in Camunda Operate, so users can see at-a-glance the progress of any process.
In addition to supporting and operating a highly flexible DevOps philosophy, Camunda has enabled Deutsche Telekom IT to visualize complex logic in one place, easily align human and automated tasks, and use the same BPMN language for business and development.
For those who are thinking of following in Deutsche Telekom IT’s footsteps, Tüting has
sage advice: “Choose your stack wisely. Don’t start with a full stack; choose components that add the most benefits to your needs. Think big, start small, don’t try to get to the holy grail in the first attempt – evolve to where you want to be.”
Case Study: Migrating a major financial services firm step by step
Consulting firm Doculabs works with many large financial services organizations to modernize their IT systems. Many of these firms have built process automation on top of monolithic automation platforms. As solutions grow with time, they become brittle and difficult to change. One major financial services firm wanted to overhaul their front office and back office processes — from opening new accounts and moving money to compliance workflows. Ideally, they wanted to move to a cloud-based process orchestration platform with proper security and governance controls in place.
The first step was to inventory each business process across the firm and place each one in a “small,” “medium,” or “large” bucket. Sizing processes in this way required teams to review the rules, decision points, integrations, and number of reports involved in workflows, as well as the approximate cost of each workflow. The team decided that the biggest initial wins would come from rebuilding the highest volume process types from scratch. In contrast, the team estimated that many of the lower-volume processes that are used on a day-to-day basis would require the most time and effort to migrate. Dormant processes with little to no volume were retired.
From there, the Doculabs team built a business case for migrating away from the automation monolith. They positioned the migration as part of a larger modernization effort and discussed the benefits of a modern business process automation platform for both end users (consumers) and for employees at the company. With a new, Camunda-based solution, the team would be able to change how the firm services customers, reduce the amount of manual work involved in key processes, and complete workflows faster.
After gaining initial buy-in from the financial services firm’s leadership, the Doculabs team planned PoCs throughout the organization to prove how the new approach would work. The Doculabs team worked to understand the process inventory, dissect and prioritize each workflow, and move each process to the new environment in priority order. Using the PoC approach, they experimented with pulling workflows from the monolithic system and migrating them to the Camunda-based platform. They also re-engineered certain workflows before migrating them to the new environment.
At a high level, the team worked as follows:
- If a process is broken or the technology is inadequate, re-engineer and rebuild the process
- If a process is working but the technology is inadequate, rebuilt the process; simplify it, standardize it, and reuse it wherever possible
- If a process is working and the technology is adequate, migrate the process as-is (as much as possible)
After Doculabs migrated the firm’s processes from their old monolithic automation platform to Camunda’s cloud-based platform, they saw an immediate improvement in collaboration between business and technical teams. Using BPMN as a common language and communication tool, both IT and business teams could easily see how processes were working and how they could be improved. The new platform also enabled the firm to orchestrate automated tasks and work done by people together in a cohesive way.
Camunda Best Practices Hub
Are you ready to start your migration? The Camunda Best Practices Hub provides detailed advice on choosing a tech stack, modeling business processes, executing a PoC, and more.
Case Study: Lessons learned from a successful migration journey
Swiss Reinsurance Company (Swiss Re) successfully migrated from a monolithic automation platform to microservices-based process orchestration using Camunda. Here are some of the key lessons they learned from their migration:
- Be persistent. It’s not enough to share information about the migration during a single advisory board session and expect people to understand everything. The goal is to get all of the key stakeholders to understand the dependencies and timelines associated with the migration. That happens with persistent communications.
- Try to understand the priorities of business stakeholder teams. Every team has its own projects and priorities. Aligning them at the outset of the migration process helps with the planning process.
- Be optimistic about trusting the development team. Let go of old paradigms and assumptions. Trust that your team can transition to new ways of working. In Swiss Re’s case, dedicated business process management (BPM) developers transitioned to Camunda and were eager to learn new cloud-native development skills.
- Be pessimistic about planning. Swiss Re’s migration was stalled by the COVID-19 pandemic and personal matters within the team. In other words, expect the unexpected.
- Be realistic about limitations. That means setting priorities for what can happen right now, and what needs to happen in the future. For example, Swiss Re has built a future roadmap to self-heal applications, widen its user base, and allow super-users to build processes in Camunda. In addition, they want to enhance reporting capabilities so users can understand where processes need improvement.
Conclusion
With Camunda, you can automate and optimize processes with greater agility, more transparency, better control, and lower total cost of ownership. Camunda’s end-to-end process orchestration platform helps you overcome the challenges of automation monoliths and provides the agility you need to evolve your enterprise architecture for tomorrow.
If you’ve made the decision to migrate from a monolithic automation platform to Camunda, congratulations! Modernizing these rigid systems to a composable process orchestration solution such as Camunda can lead to better customer experiences, improved team productivity, and faster time-to-market.
If you need help along the way, reach out to Camunda Consulting. We’ve helped many organizations across industries migrate from monolithic automation platforms to Camunda. We’re happy to help steer your team through the entire migration process, from PoC to success in production.
