What’s New In Zeebe: Scaling Zeebe, New Client APIs, Faster Requests, Timestamps, Node.js Client, and Default Topic is Back!

Welcome to the first-ever edition of “What’s New In Zeebe”, where we share our progress on the journey of building Zeebe, the world’s first high-throughput, resilient, and horizontally-scalable workflow engine.

We might not publish a “What’s New In Zeebe” post every non-release week, but when there are exciting developments to share in between releases, we want to be sure we’re getting the word out.

If you have questions for the Zeebe team, or you’d like to get involved with the project, take a look at the Zeebe Community page. There, you’ll find our user support forum, public Slack channel, the Zeebe GitHub project, and our issue tracker. We gladly welcome new users and contributors to the Zeebe community.

Scaling Zeebe

Felix has been running tests on AWS, particularly to test the new clustering layer released in 0.9.0. Apparently, Zeebe scales 🙂

Zeebe Horizontal Scalability Testing

This is a great milestone for Zeebe because prior to introducing the replication factor, each broker would replicate each partition, so adding more brokers would not allow you to scale throughput–it would only make the system more resilient by having more copies of the data.

New Client APIs

Thorben and Philipp pushed the first increment of a new Java API that’s a lot easier to use and makes it clearer to users how to formulate fluent API requests like this:

final ZeebeFuture<WorkflowInstanceEvent> workflowInstanceEvt = zeebe.workflowClient()
    .newCreateInstanceCommand()
    .bpmnProcessId("foo")
    .latestVersion()
    .send();

Note how it always returns a future that you can decide to block on or not.

There is a lot more to the new API, and I am sure that the team will provide us with a more detailed update after the next alpha release.

Sam also adjusted the Golang client to the new protocol, and we can expect full support for it in the next release.

Faster Requests

We have also been working on making the transport layer faster and scale better when communicating with multiple brokers from one client or broker. The results are great:

New Zeebe Client APIs

You can find more details in this GitHub issue.

To achieve this, we rewrote the buffer management in the transport layer, and now it is also possible to send requests asynchronously in a loop without having to wait for the response.

Felix is particularly happy that he can write his performance tests like this:

for (int i = 0; i < 1_000_000; i++)
{
    zeebe.workflowClient()
      .newCreateInstanceCommand()
      .bpmnProcessId("foo")
      .latestVersion()
      .send(); // async sending of the request
}

Timestamps

Thanks to Nicolas, each record in Zeebe now has a timestamp. Timestamps have millisecond precision and reflect when the record (event or command) was appended to the partition / log.

This work prepares us for upcoming support for Zeebe in Camunda Optimize and another webapp we are going to build but that is still “on the QT and very hush hush”…

Node.js Client

Sebastian, Sam, and Felix have made great progress on exposing the Golang client over FFI so that it can be used as a shared library in an upcoming Node.js client.

As far as I hear, there was a major breakthrough and both Request-Response but also Subscription Style interactions work.

Work is being conducted here: https://github.com/zeebe-io/

Default Topic Is Back!

It is now possible to define one or multiple topics along with partition count and replication factor in the configuration file:

[[topics]]
name = "development"
partitions = 3
replicationFactor = 1

[[topics]]
name = "production"
partitions = 16
replicationFactor = 3

Also, the broker will ship with a default topic configured out of the box, so no more creating a topic in each demo or unit test that you write. I know a certain guy who will be very happy about that!

If you’d like to stay on top of what’s new with Zeebe, you can subscribe to our newsletter or follow us on Twitter.

  • Title slide that reads "Why Camunda 8"

    Why R-KOM Chose Camunda Platform 8

    In this blog series, we highlight the customers who have chosen to utilize Camunda Platform 8 and explore the challenges those companies are attempting to overcome using process orchestration. For the latest installment of Why Camunda 8, we spoke with R-KOM, a telecommunications company based in Regensburg, Germany. When R-KOM was founded in 1997, its shareholders pooled their telecommunications infrastructure, which had evolved over decades with utility networks for water, electricity, and gas. Initially, R-KOM’s services were limited to business and the public sector, but now it has developed further in line with demand. Over the years, the company’s high-performance infrastructure and a broad range of products have grown. Today, R-KOM has a number of city networks in Eastern Bavaria...

    Read more
  • Title slide that reads "Why Camunda 8"

    Why Gruner + Jahr Chose Camunda 8

    In this new blog series, we explore the reasons why customers are migrating to Camunda 8. For our first installment of Why Camunda 8, we talked to Gruner + Jahr, one of the largest premium magazine publishers in Europe. G+J includes such established (German) print and digital brands as STERN, GEO, BRIGITTE, ESSEN & TRINKEN, and SCHÖNER WOHNEN, as well as younger brands such as CHEFKOCH, BARBARA, BEEF, 11FREUNDE. In addition to the numerous print and digital media offerings, G+J offers products and licenses such as the SCHÖNER WOHNEN collection. The digital business contributes more than half of revenues and is exhibiting continued strong growth. Indeed, the company’s digital products lead the rankings in all publishing segments, from news through...

    Read more
  • Camunda Platform 8.1.0-alpha3 Released

    We’re excited to announce the release of Camunda Platform 8.1.0-alpha3. If you’d like to get started with Camunda Platform 8.1.0-alpha3 right away, you can sign up for a free trial now. Create Process Instance Starting at User-Defined Elements An often requested feature is now available as a preview with Camunda Platform 8.1.0-alpha3: create a process instance starting at user-defined elements. When creating a process instance through the CreateProcessInstance RPC, the process instance is started at the default none start event. For testing purposes, you may want to start at one (or multiple) of the other elements. This feature is now available through both the CreateProcessInstance RPC and the CreateProcessInstanceWithResult RPC. It is available for use in the Zeebe Java client...

    Read more

Ready to get started?

Still have questions?