Gartner® recently published a report called Essential Patterns for Data-, Event- and Application-Centric Integration and Composition (9 March 2023). According to Gartner: “Today’s complex and evolving integration use cases demand a thorough understanding of both emerging and proven integration patterns. Application technical professionals should use this research to learn and use the optimal patterns to connect and compose enterprise systems, solutions and processes.
These integration patterns have three broad use cases:
- Executing multi-step business processes
- Enforcing consistency of data between databases and/or applications
- Providing services that consume APIs and/or data from other applications
The report identifies several “application-centric integration patterns for Multistep Processes,” according to Gartner, “Use patterns in this section when you want to compose a workflow or process from data, tasks or capabilities provided by other applications, services or subprocesses. The new workflow created here implements a business process that can function as an independent application and may span multiple domains.”
Let’s take a look at the three application-centric patterns that Gartner describes.
The process orchestration pattern
Process orchestration coordinates the various moving parts (or endpoints) of a business process, and sometimes even ties multiple processes together. A process orchestrator centralizes and controls the process flow; Gartner points out that “because the flow control is centralized, it’s relatively simple to monitor, optimize and modify the orchestrated process in one place.”1
Process orchestration is an integration pattern that has emerged as more and more companies across industries have digitally transformed their business. The roots of process orchestration lie in the world of business process management (BPM). That said, BPM was originally intended for stable, repeatable, back-office operations. Process orchestration is a more sophisticated integration pattern that’s better suited for the critical processes that build and sell a company’s products and/or services.
The Gartner report says that process orchestration is “suitable for complex and frequently modified process flows that involve many (more than three) participants.”1 In the 2023 State of Process Orchestration Report, IT decision makers who are involved in process automation shared that their business processes require an average of 5.68 participants (also known as endpoint types), so orchestration is a suitable pattern for designing and running those processes.
The process choreography pattern
According to Gartner, process choreography “takes a decentralized approach to compose services, where the choreography logic is distributed and embedded within each participating service. The developers of the choreography need to craft the individual service’s behavior or logic such that they collaborate and coordinate collectively to achieve the desired process outcomes.”1 There is no centralized orchestrator that tells the components of a process when to interact or how to do so. Instead, the process is distributed across the tools that are used to execute each step. Generally speaking, process choreography requires an event broker such as Apache Kafka to facilitate communication between process participants.
Choreography is a popular choice for microservices environments. It promotes loose coupling between individual services, reducing dependencies and making it easier to replace services without changing the whole process. Services interact with one another through predefined messages or events rather than relying on direct method invocations or tight integration. However, the downside is that without a centralized process orchestrator, it can be hard to visualize, understand, monitor, and manage interactions between microservices. When it comes to business-critical processes, these are crucial capabilities.
Garter says that process choreography “is suitable for simple transaction flows that involve a small number of participants when process monitoring service is not implemented.”1 We recommend considering how you can mix choreography and orchestration to gain the benefits of both.
The human-in-the-loop pattern
A human-in-the-loop (HITL) workflow is just what it sounds like: a business process that requires people to execute tasks. HITL workflows are partially automated, often long-running, and usually rely on people for some part of the process, whether that’s escalation, mitigation, and troubleshooting.
RPA tools are often used to automate individual steps in HITL workflows for back-office operations, especially those that involve paper documents. For example, in a reimbursement process that requires employees to submit a PDF of an invoice, an RPA bot can facilitate the workflow by scanning the PDF and extracting data needed for reimbursement (such as the purchase date and amount). When it’s time for an accountant to review the employee’s request, they only need to confirm that the bot captured the data correctly.
But while RPA tools are useful for HITL workflows, they lack some of the key characteristics that the Gartner report highlights. These might include “long-running, multiparticipant business processes that require you to persist states.” Processes that involve people are often measured in minutes, hours, or even days. It’s important that these processes flow properly, even if the process has to wait while a person completes a task. RPA bots don’t have the capability to persist the state of a process instance over a long period of time.
Additionally, Gartner notes that one of the key characteristics for HITL workflows is “built-in monitoring of metrics for process execution times, and provides real-time visibility of technical and business metrics gathered.” While RPA tools offer monitoring and metrics for automated tasks, they don’t provide the combined view of automated work and human work that you can get with a solution that executes the business process from start to finish. Even in a digitally transformed business, human work is often the best place to find opportunities for improvement, so having hard data to analyze HITL workflows is vital.
Which pattern is right for me?
To quote Camunda’s Chief Technologist, Bernd Ruecker: “It depends!”.
- Process orchestration is appropriate for complex processes with more than three participants (RPA bots, CRM or ERP tools, legacy systems, IoT devices, etc.)
- Choreography is appropriate for simple transaction flows with less than three participants (and remember that choreography can be combined with orchestrated processes to gain the benefits of both)
- Human-in-the-loop (HITL) flows are appropriate for processes that mix automated tasks with work done by people, and provide opportunities for increased automation
Get a taste of process orchestration with Camunda Platform 8
End-to-end orchestration of business-critical processes is Camunda’s jam. Try it yourself by signing up for a free 30-day trial of Camunda Platform.
1GARTNER®, Essential Patterns for Data-, Event- and Application-Centric Integration and Composition, 9 March 2023. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved
