How to Write Glue Code without Java Delegates in Camunda Platform 8

*Camunda Platform 8, our cloud-native solution for process orchestration, launched in April 2022. Images and supporting documentation in this post may reflect an earlier version of our cloud and software solutions.


Introduced in 2015, the external task pattern is on the rise. Instead of the workflow engine actively calling some code (push), the external task pattern adds the work in a sort of queue and lets workers pull for it. This method is also known as publish/subscribe. The workflow engine publishes work, and workers subscribe to be able to do it.

Within Camunda Platform 7, we work on making the external task pattern the default recommendation, and for Camunda Platform 8, this is the only way of writing glue code. Specifically, JavaDelegates are not possible in Camunda Platform 8 anymore. This sometimes seems to leave people puzzled, so this blog post will answer why this is not a problem, and dive into the benefits you can gain from external tasks. This blog post will also debunk some myths about this pattern and clarify that: You can still call your service endpoints via any protocol (e.g. REST, AMQP, Kafka).

  • You can have all worker code in one Java application.
  • The programming model looks surprisingly similar to JavaDelegates when using Spring.
  • Exception handling can still be delegated to the workflow engine.
  • The performance overhead in terms of latency is small.

Architecture considerations

Let’s debunk two architectural myths of external tasks.

First, applying external tasks does not necessarily mean services you formerly called via REST now need to fetch their work themselves. While this is an architectural option, it is not the typical case. Let’s look at an example:

You likely will implement a worker that still does the REST call towards the payment microservice (left side of the illustration above). This is the API the microservice exposes and should be used. The worker is in the scope of the order fulfillment process solution or microservice. Nobody outside of the order fulfillment team even needs to know that Camunda or an external task worker is used at all.

Compare that to the solution on the right side of the example, where the payment microservice directly fetches its work from Camunda. In this case, Camunda is the middleware used for various microservices to communicate amongst each other. While this is feasible and has its upsides, I have rarely seen it in the wild. Read more on further discussion of the differences

The second myth is that you have to write multiple applications if you have multiple service tasks, one for every external task worker. While you can separate workers into multiple applications, it is rare. It is much more common to run all (or at least most) of your workers in one application.

This application belongs logically to the process solution and registers a worker for every external task. This process solution can also auto-deploy the process model.

Writing glue code

This leads us to the question of how to write glue code. Within that realm, there is another myth: it must be complicated because there is remote communication involved. The good news is that this is not necessarily true for Camunda Platform 8, as there are programming language clients that provide a great developer experience. For example, using the Spring integration, you can write worker code like this:

@ZeebeWorker(type = "payment", autoComplete = true)
public void retrievePayment(ActivatedJob job) {
  // Do whatever you need to, e.g. invoke a remote service
  String orderId = job.getVariablesMap().get("orderId");
  paymentRestClient.invoke(...); 
}

If you compare this code to a JavaDelegate, it looks surprisingly similar. We even created a community extension containing an adapter to reuse existing JavaDelegates for Camunda Platform 8. While I would not necessarily recommend doing this as it’s better to migrate your classes manually, it nicely shows that this is conceptually not too hard.

That said, there are some things you can do in JavaDelegates that are no longer possible in external task workers:

  • Access workflow engine internals
  • Influence the workflow engine behavior
  • Integrate with thread pools or transactions of the workflow engine
  • Dirty hacks using Threadlocals or the like

In general, I feel it’s a good thing you cannot do these things anymore, as they regularly lead teams into trouble. 

Note that we are also working on increasing the convenience of external tasks with Camunda Platform 7, and just started this community extension.

Handling exceptions

When writing glue code, you can also pass problems within your worker to the workflow engine to handle them. For example, the workflow engine can trigger retrying or raising an incident in the operations tooling. The code is pretty straightforward, and yet again quite comparable to JavaDelegates:

@ZeebeWorker(type = "payment", autoComplete = true)
public Map<String, Object> retrievePayment(ActivatedJob job) {
  // some work
  // any exception thrown here will lead to retrying/incidents in Zeebe
  if (successful) {
    // some data is returned to be stored as process variable
    return variables;
  } else {
   // problem shall be indicated to the process and can be handled via BPMN error events
   throw new ZeebeBpmnError("DOESNT_WORK", "This does not work because...");
  }
}

Read more on this in the spring-zeebe docs.

However, there is one important failure case not yet sufficiently handled: what if a worker crashes and does not fetch any work anymore? Currently, Camunda Platform 8 recognizes this indirectly by service tasks not being processed for too long. Ideally, the workflow engine itself should recognize that work is no longer fetched. Then, it could indicate this in the operations tooling. We are currently looking into this feature. So long you can rely on typical systems monitoring to detect a crashed Java worker application.

And transactions?

Similarly, how do you achieve consistency between your business code and the workflow engine? What if any of the components fail? With JavaDelegates, many users delegated these problems to either transaction manager, often without knowing it. Please refer to the blog post about achieving consistency without transaction managers for how to handle this with external tasks, but also to understand why this is a preferable mental model today.

Latency of remote communication

One last myth I want to address in this post is that remote workers need to be “slow.” Often in such discussions, slow is not further defined, but looking at Camunda Platform 7, depending on the configuration of the job executor, it really can take seconds for an external task to be picked up (which can be optimized by the way). In Camunda Cloud, the whole interaction is optimized from the ground up so that only a bit of latency for the remote communication is added. In a recent experiment, I measured the overhead of a remote worker to be roughly 50ms:

In most projects, this is not a problem at all, especially as it does not affect the throughput of the workflow engine. In other words, you can still process the same number of process instances, they simply require 50ms longer per service task. Note that we are further optimizing this number for low latency scenarios we are seeing among customers.

Summary

As you can see, you have a programming model that is as convenient as JavaDelegates. At the same time, you have code that is properly isolated from the workflow engine (moving from embedded to remote engines dives into all advantages of the remote engine setup).

This is why I am personally so excited about switching to external tasks as our default glue code pattern. If you are not convinced or still have questions, please reach out to me or ask in the forum at any time.