Inside Zeebe

The upcoming 0.22 release of Zeebe includes a feature many users have been asking for: the ability to start a workflow and retrieve its outcome with a single command. The new gRPC command CreateWorkflowInstanceWithResult is available for testing in the current SNAPSHOT Docker image of Zeebe and the zeebe-node-next version of the Node.js client. This command starts a workflow instance and returns the outcome when the workflow completes. Use-cases A common scenario is to start a workflow in response to a REST request, and send back the outcome from the workflow in the REST response. Previous implementations of this relied on a polling worker to retrieve the outcome, and the use of subscriptions to correlate it with the REST request...

The Zeebe team just implemented a Kafka Connect Zeebe connector. This is a feature users have been asking for since the start of the project, so let’s give it a closer look. What is Kafka Connect? Kafka Connect is the ecosystem of connectors into or out of Kafka. There are lots of existing connectors, e.g. for databases, key-value stores or file systems. So for example you can read data from a RDMS and push it to Elasticsearch or flat files.

The upcoming 0.21 release of Zeebe includes a feature many users have been asking for: long-polling for workers. And make sure you stay tuned to the end to find out what a massive deal it is. Zeebe is a radical re-imagining of the workflow engine for the modern world: it uses event sourcing to interpret workflows over immutable streams. In the Zeebe model, workers are de-coupled from the broker. Conceptually, workers “subscribe” to task types on the broker to service. In terms of actual implementation, workers poll the broker for the task type they service. This allows the broker to have no stateful knowledge about workers. The Problem With Polling Every request to the broker for work from a worker...

In early July, 2019, I sat down with Daniel Meyer, Camunda CTO and Zeebe Committer #0, to talk about the new Zeebe Community License. This is the transcript of that conversation. Josh Wulf: We have a change to the licensing for Zeebe – probably, people are going to be interested to know more about that. I know you’ve done a lot of work on it – you’ve been speaking with British lawyers most recently about it. Maybe you can give a bit of background on the thinking behind that. Daniel Meyer: Right, of course – happy to. And let’s maybe start by just reflecting on the tremendous journey that open source had over the last twenty years plus. So, I...

Last Wednesday, July 17, we announced the first production ready release of Zeebe, Camunda’s new cloud-native workflow engine for microservices orchestration. Zeebe is a new code base, written from scratch, putting forward a completely new way of architecting workflow engines for microservices and distributed systems. This blog post is a quick overview of why and how we did that.

Please note that this blog post has not been updated since its original publication to reflect changes to Zeebe’s gateway.proto file. The general approach for creating a stub that we show here still applies to newer versions of Zeebe, but you may need to make a few modifications depending on which Zeebe version you’re using. The high points: Starting in Zeebe 0.12, Zeebe clients communicate with brokers via a stateless gRPC gateway, with Protocol Buffers used as the interface design language and message interchange format. gRPC makes it easy to generate a “client stub” in any of ten supported programming languages; this means it’s possible to use Zeebe in applications written in not only Java and Go–the languages with officially-supported...

Zeebe 0.12 supports message events, making it easy to use Zeebe as part of an event-driven architecture; workflow instances can to respond to events published by external systems. And Zeebe handles all of the heavy lifting related to correlating a message to a workflow instance: buffering incoming messages, opening a subscription to receive messages that match with active workflow instances, and correlating a message (when one exists) to a workflow instance that has arrived at a message event. Last month’s Zeebe 0.12 release introduced support for two BPMN elements that are core to Zeebe’s ability to operate in an event-driven architecture: the intermediate message catch event and the receive task. These elements make it possible for workflow instances to react...

It’s been a busy past month for the Zeebe team here at Camunda. At the end of September, we took the stage at CamundaCon, our annual user conference, and in mid-October, we traveled to San Francisco to give a presentation at Kafka Summit.