Job Executor Improvements in Camunda BPM 7.4

If you’ve used Camunda BPM you might have noticed our engines briefest feature – it’s really really fast. Might be an idea to have a look at this whitepaper or read about our scalability if you haven’t experienced it for yourself. With the new job prioritization feature and the exponential backoff included in the new Camunda BPM 7.4.0- release you will now be able to improve the speed further by tuning the efficiency of your custom jobs among others.

Imagine you have a lot of jobs (maybe 50,000) on a cluster environment which wait to be executed all at the same time. The current behavior of the engine (Camunda BPM 7.3.0) is to acquire all these jobs in parallel and without any order. So it is possible that the engine executes a rather unimportant “historization” job first while a job that is mission critical for your business is left unprioritized.

To make execution more efficient for your specific environment and to bring order into this situation of high job executor load, Camunda 7.4.0 now enables you to set a priority for each job. This can be done very easy within the Camunda Cockpit.

Change Job Priority in Cockpit.

Furthermore with the new release it’s also possible to set job priorities dynamically for each process instance via [process variables] (https://docs.camunda.org/manual/latest/user-guide/process-engine/the-job-executor/#priorities-at-the-process-level).

At runtime the Camunda engine then evaluates the priority of each job like shown in this BPMN-diagram:

Calculate Job Priority

But what if you have a large number of jobs with the same priority and same due date?

As already shown in Thorben’s blogpost the current way of executing multiple jobs in parallel is not be very efficient. Imagine you have 4 engines in the cluster and 50,000 jobs to be executed exactly at midnight. All the engines start acquiring jobs exactly at midnight, but only one of them locks the first 50 jobs exclusively and starts executing them. The three remaining engines can’t get any job, so 3/4 of an acquisition cycle is wasted without executing jobs.

To handle this inefficient behavior, the new Camunda BPM 7.4.0-release provides a completely new approach, the exponential backoff. The engines in your cluster tries no longer to acquire jobs in parallel, but now they try it sequentially with a short delay (maybe 50 to 150 ms, the backoff). In consequence each node will only pick jobs that have no exclusive locks of other engines. So from now on, the acquiring jobs have only minimal overlap (if any) and all the jobs can be executed as quickly as possible.

For further details please have a look at the reference or user guide to find out how to configure your engine in very heavy load scenarios.

  • Monitoring Camunda Platform 7 with Prometheus

    Monitoring is an essential facet of running applications in a production system. Through this process, organizations collect and analyze data, and determine if a program is performing as expected within set boundaries. When combined with alerting, monitoring allows for detecting unexpected system behavior to mitigate exceptional situations as fast as possible. Furthermore, tracking the performance of a system enables organizations to improve those aspects that have the biggest impact with higher priority. One essential aspect of monitoring is the list of key metrics you want to observe. There are different categories of statistics that can be of interest here. To observe the defined metrics, there are plenty of application monitoring tools on the market today. They differ in many aspects...

    Read more
  • Securing Camunda 8 self-managed cluster and applications...

    Directory services are an effective way to manage an organization’s users, groups, printers, devices, and more. Most organizations accomplish this using Active Directory, Apache Directory, Oracle Internet Directory, or other similar tools. Recently I worked with a customer who wanted to see how he could secure the Camunda 8 Platform and process applications with such a directory. Their requirements consisted of: Allowing Directory users to access Camunda applications (Tasklist, Operate, Optimize) Accessing secured Tasklist & Operate APIs from our custom project Securing the custom project In this article, I’ll briefly explain the 3 easy steps taken to fulfill their requirements which include: Federate users from the Directory service into Keycloak Declare an application in Identity to access Camunda APIs Configure...

    Read more
  • Accelerate Connectivity with Camunda Platform 8.1

    We’re thrilled to announce Camunda Platform 8.1, the latest release of our process orchestration solution. This new version introduces features that accelerate connectivity to the many different systems and technologies that are required for true digital transformation, including: Create custom Connectors with our Integration Framework to accelerate connectivity New out-of-the-box Connectors for popular services Enhancements to Camunda Modeler that improve productivity Hot backups and official support for Amazon EKS and Red Hat OpenShift Plus, several upgrades requested by Camunda Platform 7 customers Organizations across all industries rely on complex technology stacks to adapt and enhance their operations in response to market dynamics, new disruptive companies, and increasing consumer expectations. Your technology stack likely includes everything from cutting-edge technologies to legacy...

    Read more

Ready to get started?

Still have questions?