Camunda supports process automation at scale

How Camunda 8 supports process automation at enterprise scale

Scaling process automation across an enterprise naturally includes a wide array of challenges.

To counter these challenges, more and more organizations are evolving their organizational structures accordingly, often creating Centers of Excellence (CoE) and/or communities of practice (CoP). Please note that during this post we will use the term CoE, but keep in mind your organization might call this very differently. That said, in Camunda Platform 8, we included new functionalities that will help companies to scale process automation successfully. 

This blog post discusses different aspects of scaling process automation, the role of CoEs, and how new Camunda 8 features support those initiatives. 

What does scaling mean?

Organizations that want to create a competitive edge through digital transformation (and frankly: which don’t?) need to figure out how to scale process automation.

Scale in this context has two main dimensions: 

  1. Technical scale: The technical capability to run more load, which is for example supported by horizontal scalability of Zeebe, the workflow engine within Camunda Platform 8. 
  2. Organizational scale: The organizational ability to implement, maintain and operate an increasing amount of automation solutions across the enterprise, while also leveraging economies of scale.

While the technical abilities are a crucial foundation for scaling process automation, it’s actually organizational aspects that can determine the success or failure of strategic automation initiatives.

Accordingly, as a recent survey by McKinsey points out, the more parts of the organization are involved, the more automation initiatives are likely to succeed which. Other parts of the organization might include things like as different stakeholders and business domains. In Goldman Sachs’ talk at CamundaCon 2020 , for example, they call this “process automation at enterprise scale”. Our post focuses on these organizational challenges of scaling process automation and how Camunda 8 can help here.

Dimensions of Organizational scaling and its challenges

Organizational scaling includes multiple dimensions. For this post, we want to focus on the two most important: 

  • Dimension 1: Multiple process automation projects are running in different parts of the organization — meaning that more and more people and teams will be facing similar questions and challenges, such as architecture decisions, or developing the necessary skills (e.g. for modeling BPMN). 
  • Dimension 2: Every automated process will touch multiple services handled by different teams or business units, potentially resulting in a multitude of workflow engines being deployed across the enterprise. This might look like sketched in the illustration below.
Scaling Camunda engines in your organization

How CoEs help overcome automation challenges 

To counter these challenges, organizations are more frequently establishing CoEs.

A staggering 91% of all respondents of our state of process automation report 2020 reported that they either already have a CoE in place or are actively working or planning on implementing one. According to the McKinsey survey mentioned earlier, companies leading in automation meanwhile follow a trend towards a federated CoE governance. As the analysts point out: “In this model, a central support team provides the necessary tools and capabilities to enable parts of the business to automate processes autonomously.” 

Correspondingly, key tasks that we’re seeing for CoEs are:

  • Communication, e.g. evangelizing for automation in the organization
  • Enablement, e.g. providing best practices and build skills for automation teams
  • Governance, e.g. providing getting started packages, frameworks and architectural guidelines for automation teams, without restricting them

It is important to highlight that CoEs should not provide overly restrictive guidelines and become too bureaucratic, or spend too much time with initiatives that don’t generate value for automation teams. If set up properly, a CoE will be instrumental in scaling process automation while generating excitement and buy-in.

Camunda 8 helps to scale process automation 

With Camunda 8, we improved support for scaling process automation across the enterprise, which will also help CoEs do their job. This is mostly about workflow engine provisioning and governance, simplifying collaboration between stakeholders using process models, accelerating implementation, and allowing to reuse connectors to third-party systems.

Let’s go over this one by one.

Engine provisioning and governance

With Camunda 8, we introduced a lot of developer convenience around remote workflow engines, making it much easier for teams to use them. At the same time, this setup allows separation of the development of the process solution from the actual configuration and operation of the workflow engine. Now that dedicated teams take care of running the workflow engine, others can focus on their key tasks: Implementing and delivering automation projects to solve business problems.

In the most efficient cases, the hosting is outsourced to Camunda itself. Project teams can simply use the Camunda SaaS offering. Nobody has to worry about how to install, configure, or secure Camunda. You can easily see all workflow engines running for your organization, the Camunda version they run on, their health status, and various metrics like instance counts or incidents. In this setup, you can trigger updates on the workflow engine version without even touching the automated process itself. You could set up generic alarms for technical incidents or service level agreements (SLA) for all process solutions. This will all increase the availability and quality of your automated processes and helps fundamentally with the governance challenges of scaling process automation.

If using a SaaS model is not possible, your CoE can also set up a comparable in-house SaaS model. This allows your teams to centralize knowledge on how to install Camunda, which Kubernetes cluster to use, what Elasticsearch to connect to, or how to apply a helm chart, so that development teams don’t have to build up this infrastructure knowledge. Ideally, the CoE offers the workflow engine capability as a self-service to your company. 

Simplified collaboration using Web Modeler

The Camunda Web Modeler lays the groundwork for deep collaboration between business and IT.

You have a lot of features at your disposal to make BPMN diagrams the centerpiece of communication in your process automation projects. For example, you can directly use comments in BPMN elements to synchronously or asynchronously discuss with the different stakeholders. All diagrams can be easily shared with project teams or individuals. They can further be embedded anywhere in a read-only mode, for example, your confluence or documentation portal, which is especially useful if you need to involve a lot of stakeholders in an enterprise context. These collaboration capabilities further improve business IT collaboration through BPMN, as well as the time to value for automation projects.  

For us, the current state of Web Modeler is a solid starting point and the target state promises even more. We plan to go further by providing simulation or debugging capabilities, and link in insights from other parts of the platform, e.g. around deployed versions, current performance of the process in production, incident status, or the like. 

Solution acceleration and connectors

One key success factor to scale process automation is to speed up the time to deploy any new project into production. This can be done in many ways, like for example, using industry best practices from software engineering around continuous delivery. But there are also specific areas, where a process orchestration solution like Camunda can help accelerate projects.

Besides the many small things we do to enhance developer productivity (e.g. the upcoming expression designer, or debugging capabilities), we also reinvented connectors for Camunda 8 with what we call the integration framework.  A connector is a component that talks to a third-party system via an API and thus allows orchestrating that system via Camunda (or lets that system influence Camunda’s orchestration). 

The connector consists of a bit of programming code needed to talk to the third-party system and some UI parts hooked into Camunda Modeler. Because we define a software development kit (SDK), anybody can easily develop their own connectors, that then will be usable in the Camunda stack (see this technical sneak peeak into Camunda’s Connector architecture for more technical details).

This means that there will be out-of-the-box connectors available for standard problems (currently we started with REST, Sendgrid, and Slack), either provided by Camunda, partners, or the community. Those will make it easy to connect to certain systems. Using connectors is always optional, and connectors will mostly be available open source and thus be extensible, so it does not get in the way of the developer-friendliness we value so much.

The integration framework also means that organizations, and possibly their CoEs, can implement their own connectors to be reused internally. Experience clearly shows that there are typically a small number of legacy systems and a lot of processes have to be connected to. Prebuilding that logic and making it available for re-use can significantly accelerate process automation projects. 

Some organizations might even use this, to enable less technical roles – like citizen developers – building process automation solutions, which can free up IT capacity, further improving time to value for automation projects. In such a scenario, the Web Modeler can be used not only for modeling BPMN, but also to configure predefined connectors through the new properties panel to implement a fully executable process. 

Intelligence

Process intelligence is instrumental when establishing process automation within organizations, as corresponding tools allow ever closer involvement of business stakeholders in projects that are very often IT-heavy. 

The possibility to see real-time business key performance indicators (KPIs) from runtime engines and processes strongly underlines the business case of automation projects. This helps business teams and CoEs to showcase and communicate the ROI of their automation initiatives. Additionally, it makes it easy to identify process inefficiencies or bottlenecks, allowing to continuously improve automation projects. This approach helps you to build trust and awareness around the advantages of process automation tooling.

With Camunda 8, we continue to improve Optimize, our process intelligence tool, and make it especially easier to create helpful analysis and actionable insights out-of-the-box. 

Conclusion

We looked at the aspects of scaling process automation and particularly focused on the corresponding organizational challenges and the role of CoEs in this context. With the aim of supporting our customers in implementing “automation at enterprise scale” lead to the release of these new Camunda 8 features discussed here.

If this is interesting to you and want to learn more, feel free to get in touch with your known Camunda contacts, or check out our other resources, such as our blog, whitepapers and events.

Ready to get started?

Still have questions?