Camunda Best Practices: Monitoring & Orchestrating Your Microservices Landscape

Back in March, I conducted the webinar: “Monitoring & Orchestrating Your Microservices Landscape using Workflow Automation”. Not only was I overwhelmed by the number of attendees, but we also got a huge list of interesting questions before and, especially, during the webinar.

I was able to answer some of these, but ran out of time to answer them all. So I want to answer all open questions in this series of seven blog posts – you can click on the hyperlinks below to navigate to the other entries.

In this blog, we’ll be exploring Camunda Best Practice questions:

  1. BPMN & modeling-related questions (6 answers)
  2. Architecture related questions (12)
  3. Stack & technology questions (6)
  4. Camunda product-related questions (5)
  5. Camunda Optimize-specific questions (3)
  6. Questions about best practices (4)
  7. Questions around project layout, journey and value proposition (3)

Q: Business data versus workflow data: if you cannot tear them apart, how can you keep them consistent? Are the eventual/transactional consistency problems simpler or more complex with Camunda BPM in the equation?

This is quite a complex question, as it depends on the exact architecture and technology you want to use.

Example 1: You use Camunda embedded as a library, probably using the Spring Boot starter. In this case, your business data could live in the same database as the workflow context. In this case you can join one ACID transaction and everything will be strongly consistent.

Example 2: You leverage Camunda Cloud and code your service in Node.JS, storing data in some database. Now you have no shared transaction. Now you start living in the eventual consistent world and need to rely on “at-least-once” semantics. This is not a problem per se, but at least requires some thinking about the situations that can arise. I should probably write a dedicated piece about that, but I had used this picture in the past to explain the problem:

At least once

In this example, you can end up with money charged on the credit card but the workflow not knowing about it. But in this case you leverage the retry capabilities and will be fine soon (=eventually).

Q: Is it a good idea to save the process data into a single complex object with JSON notation?

As always: It depends.

Our general recommendation is to have as little data as possible in a workflow, and if the data lives somewhere else, only store references.

But if the data cannot live anywhere else, serializing the data into the workflow is at least an option. And if you serialize it, JSON might also be a good idea. Please note, that you cannot query for data in that process variable anymore (or do a text query at max).

So it depends. The best practice Handling Data in Processes might help to judge that.

Q: How to handle the filtering of information allowed or not to be seen by a user of the process?

I guess this refers to task lists. There are two layers to look at it. Is it important that the user is not able to get the data at all — not even by looking at the data transferred to his browser via JSON in the background?

If yes, you need to work with data mappings to make sure only the variables are available in the task that should be readable and configure permissions accordingly.

If no, the easiest option is to simply not show certain data in the forms for users.

Q: How to do microservice versioning and workflow versioning and manage both in harmony and congruent?

I love this question, but it depends. Versioning Process Definitions might give you a good starting point for the workflow angle of it.

For the microservices angle of it there are tons of other material available discussing tolerant readers and overall versioning approaches. This is a rabbit hole I don’t want to follow here.

Ready for more?

Next week we’ll be taking a closer look at questions on project layout, journey and value proposition. But if you can’t wait until the next blog, you can check out the original here on my Medium site.

Getting Started

Getting started on Camunda is easy thanks to our robust documentation and tutorials

  • Monitoring Camunda Platform 7 with Prometheus

    Monitoring is an essential facet of running applications in a production system. Through this process, organizations collect and analyze data, and determine if a program is performing as expected within set boundaries. When combined with alerting, monitoring allows for detecting unexpected system behavior to mitigate exceptional situations as fast as possible. Furthermore, tracking the performance of a system enables organizations to improve those aspects that have the biggest impact with higher priority. One essential aspect of monitoring is the list of key metrics you want to observe. There are different categories of statistics that can be of interest here. To observe the defined metrics, there are plenty of application monitoring tools on the market today. They differ in many aspects...

    Read more
  • Securing Camunda 8 self-managed cluster and applications...

    Directory services are an effective way to manage an organization’s users, groups, printers, devices, and more. Most organizations accomplish this using Active Directory, Apache Directory, Oracle Internet Directory, or other similar tools. Recently I worked with a customer who wanted to see how he could secure the Camunda 8 Platform and process applications with such a directory. Their requirements consisted of: Allowing Directory users to access Camunda applications (Tasklist, Operate, Optimize) Accessing secured Tasklist & Operate APIs from our custom project Securing the custom project In this article, I’ll briefly explain the 3 easy steps taken to fulfill their requirements which include: Federate users from the Directory service into Keycloak Declare an application in Identity to access Camunda APIs Configure...

    Read more
  • Accelerate Connectivity with Camunda Platform 8.1

    We’re thrilled to announce Camunda Platform 8.1, the latest release of our process orchestration solution. This new version introduces features that accelerate connectivity to the many different systems and technologies that are required for true digital transformation, including: Create custom Connectors with our Integration Framework to accelerate connectivity New out-of-the-box Connectors for popular services Enhancements to Camunda Modeler that improve productivity Hot backups and official support for Amazon EKS and Red Hat OpenShift Plus, several upgrades requested by Camunda Platform 7 customers Organizations across all industries rely on complex technology stacks to adapt and enhance their operations in response to market dynamics, new disruptive companies, and increasing consumer expectations. Your technology stack likely includes everything from cutting-edge technologies to legacy...

    Read more

Ready to get started?

Still have questions?