What’s New In Zeebe: Scaling Zeebe, New Client APIs, Faster Requests, Timestamps, Node.js Client, and Default Topic is Back!

Welcome to the first-ever edition of “What’s New In Zeebe”, where we share our progress on the journey of building Zeebe, the world’s first high-throughput, resilient, and horizontally-scalable workflow engine.

We might not publish a “What’s New In Zeebe” post every non-release week, but when there are exciting developments to share in between releases, we want to be sure we’re getting the word out.

If you have questions for the Zeebe team, or you’d like to get involved with the project, take a look at the Zeebe Community page. There, you’ll find our user support forum, public Slack channel, the Zeebe GitHub project, and our issue tracker. We gladly welcome new users and contributors to the Zeebe community.

Scaling Zeebe

Felix has been running tests on AWS, particularly to test the new clustering layer released in 0.9.0. Apparently, Zeebe scales 🙂

Zeebe Horizontal Scalability Testing

This is a great milestone for Zeebe because prior to introducing the replication factor, each broker would replicate each partition, so adding more brokers would not allow you to scale throughput–it would only make the system more resilient by having more copies of the data.

New Client APIs

Thorben and Philipp pushed the first increment of a new Java API that’s a lot easier to use and makes it clearer to users how to formulate fluent API requests like this:

final ZeebeFuture<WorkflowInstanceEvent> workflowInstanceEvt = zeebe.workflowClient()
    .newCreateInstanceCommand()
    .bpmnProcessId("foo")
    .latestVersion()
    .send();

Note how it always returns a future that you can decide to block on or not.

There is a lot more to the new API, and I am sure that the team will provide us with a more detailed update after the next alpha release.

Sam also adjusted the Golang client to the new protocol, and we can expect full support for it in the next release.

Faster Requests

We have also been working on making the transport layer faster and scale better when communicating with multiple brokers from one client or broker. The results are great:

New Zeebe Client APIs

You can find more details in this GitHub issue.

To achieve this, we rewrote the buffer management in the transport layer, and now it is also possible to send requests asynchronously in a loop without having to wait for the response.

Felix is particularly happy that he can write his performance tests like this:

for (int i = 0; i < 1_000_000; i++)
{
    zeebe.workflowClient()
      .newCreateInstanceCommand()
      .bpmnProcessId("foo")
      .latestVersion()
      .send(); // async sending of the request
}

Timestamps

Thanks to Nicolas, each record in Zeebe now has a timestamp. Timestamps have millisecond precision and reflect when the record (event or command) was appended to the partition / log.

This work prepares us for upcoming support for Zeebe in Camunda Optimize and another webapp we are going to build but that is still “on the QT and very hush hush”…

Node.js Client

Sebastian, Sam, and Felix have made great progress on exposing the Golang client over FFI so that it can be used as a shared library in an upcoming Node.js client.

As far as I hear, there was a major breakthrough and both Request-Response but also Subscription Style interactions work.

Work is being conducted here: https://github.com/zeebe-io/

Default Topic Is Back!

It is now possible to define one or multiple topics along with partition count and replication factor in the configuration file:

[[topics]]
name = "development"
partitions = 3
replicationFactor = 1

[[topics]]
name = "production"
partitions = 16
replicationFactor = 3

Also, the broker will ship with a default topic configured out of the box, so no more creating a topic in each demo or unit test that you write. I know a certain guy who will be very happy about that!

If you’d like to stay on top of what’s new with Zeebe, you can subscribe to our newsletter or follow us on Twitter.

  • Camunda Platform 8.1 Released – What’s New

    We’re extremely excited to announce the release of Camunda Platform 8.1.  In this post, we will go into the highlights of this release including Web Modeler, Connectors, Zeebe, Operate, Optimize, and more. Here’s the highlights of what’s included in this release: Easier Installation and maintenance for Camunda Platform Self-managed Hot backups for Self-Managed Token-Based authentication Modeler updates Process Templates FEEL Expression Editor Inline Annotations Changes to Camunda forms Connectors updates Workflow engine updates Operate updates Optimize updates Let’s get started! Easier installation and maintenance for Camunda Platform Self-Managed Improved Guides and Support for OpenShift and Amazon EKS (AWS) With this release, we are introducing official support for deploying and running Camunda Platform 8 Self-Managed on OpenShift and Amazon EKS (AWS)....

    Read more
  • Camunda Platform 7.18 Released – What’s New

    We’re extremely excited to announce the release of Camunda Platform 7.18 Here are some of the highlights: Cockpit: Improved instance selection to avoid unintended batch operations Cockpit: Easier navigation of high batch operation volumes MDC logging features Camunda Forms in Tasklist: Populate select options via process data Exception error codes Filter and order tasks by last updated WebSphere Liberty New and retired supported environments You can download Camunda Platform or run it with Docker. For a complete list of the changes, please check out the release notes. For patched security vulnerabilities, see our security notices. If you want to dig deeper, check out the source code on GitHub. Cockpit: Improved instance selection to avoid unintended batch operations Previously, when performing...

    Read more
  • Increase your resilience with new regions in...

    We’re excited to announce new regions for Camunda Platform 8 SaaS that further strengthen the resilience, performance, and data requirements for our SaaS customers. Enterprise customers can now choose to create their clusters in a number of regions including Europe West, US Central, US East, and the most recent addition, Australia South East. This provides multiple benefits, such as increasing availability, allowing to set up fail-safes, adhering to regulatory requirements for regional data storage, and reduced latency due to closer physical proximity. Resilience and redundancy in technology are important for all modern organizations and are top of mind for many. Across industries, organizations need a solution that scales effectively and is resilient to provide a good base for process orchestration....

    Read more

Ready to get started?

Still have questions?