Announcing Zeebe and Operate 0.23

We’re excited to announce the release of Zeebe & Operate 0.23.0 and Zeebe Modeler 0.9.0!

As usual, if you’d like to get started immediately, you can find information about it directly on the Zeebe & Operate documentation website.

Here are some highlights:

• Zeebe
  • Introduction of the FEEL expression language as a replacement for JSON path expressions in workflows
  • Adoption of Spring Boot for broker and gateway configuration, replacing the old TOML format
  • Changes to the snapshot format to ease upgradability in future versions
• Operate
  • Improvement handling instance operations
  • Highlighting of executed sequence flows
  • Automated state migration between versions
• Modeler
  • Error End Event support
  • Deploy diagram and start instances from modeler

In the rest of this post, we’ll go into more details about the changes that the latest stable releases bring.

Breaking Changes in Zeebe

It is not possible to upgrade a system with running state to Zeebe 0.23.0 from a previous version. Zeebe broker snapshots in 0.23.0 are incompatible with previous releases. You must drain the system’s running state, and stand up a new server.

When migrating to 0.23.0, you need to update and redeploy your existing workflows. We replaced JSON path expressions with FEEL. You can read more about it below, or dive into the documentation directly. A migration tool is available, which will automatically convert JSON path expressions in your existing local workflows to FEEL expressions.

What’s new in Zeebe 0.23

Expression support

The other biggest news that has us excited as part of 0.23.0 is the replacement of JSON path expressions by the expression language FEEL!

FEEL stands for Friendly Enough Expression Language, and it is a simple, powerful language that has its origin in the OMG’s DMN specification. It is designed to be side-effect free, to use a simple syntax designed for a wide audience, to provide a simple set of primitives (numbers, strings, dates, lists, and contexts), as well three-valued logic: true, false, and null.

This release introduces full support for FEEL by leveraging the wonderful FEEL Scala engine:

• Comparisons on ranges and with temporal values (e.g. date, duration, etc.)
• Calculations with numeric and temporal values
• Operations on list values (e.g. indexed access, filters, fulfillment checks, etc.)

Expressions can be used on the following element attributes (with more to come!):

• Time Catch Events: timer definition (date, duration, cycle)
• Service Tasks: job type and job retries
• Call Activities: process id of the called workflow
• Multi instance: input collection and output element
• Sequence flow conditions
• Message: correlation key

This also introduces some subtle differences from the previous JSON path expressions:

• Every expression starts with an equal sign “=” to differentiate from static values
  • Timer duration as static value: PT1H
  • Timer duration as expression: = remainingTime (i.e. the variable remainingTime holds the duration value)
• Same syntax for condition expressions as for all other expressions
  • Different syntax of the equal comparison (==) and the logical operators (&&, ||)
  • Version 0.22.2: totalCount > 10 && type == "prio"
  • Version 0.23.0: = totalCount > 10 and type = "prio"

You can learn more about the new expression support on the Zeebe documentation website.

Expression Migration

As mentioned before, this replaces our previous JSON path expressions. Existing workflows will therefore need to be migrated. To ease the migration process, there is a tool available on GitHub.

The migration can be done using a migration tool:

1. Download the JAR file from GitHub

2. Run the JAR with

java -jar zeebe-upgrade-workflows.jar <source-directory> <target-directory>

• The source directory contains the existing workflows
• The target directory will contain the migrated workflows

You can also manually migrate your workflows via the following steps:

• Add the expression prefix = (an equal sign) to every expression attribute:
  • Condition expression on sequence flows
  • Correlation key on messages
  • Input collection and output element on multi-instance activities
  • Source on input and output variable mappings
• Adjust the syntax of the condition expressions:
  • Replace == with =
  • Replace && with and
  • Replace || with or

Improved Configuration

Previously, Zeebe relied on TOML configuration files. The feedback we received was that these configuration files were not easy to generate in different programming languages, and rather uncommon in the Java world. Additionally, while there was limited support for configuration through environment variables, there were no standards, and not all settings were supported. In response to this user feedback we overhauled our configuration mechanism.

One of the biggest changes this release is the adoption of Spring Boot to bootstrap the Zeebe distribution applications, namely the broker and the gateway. While the core components of Zeebe still do not use Spring in any shape or form, the distribution adopts Spring to standardize its configuration mechanism.

What this means is that we will drop support for TOML configuration files, and from now on both the broker and the gateway can be configured via all configuration providers that Spring Boot supports: properties, command line arguments, environment variables, external YAML files, etc.

You can learn more about the new configuration system and how to migrate your existing TOML files on the documentation website.

Snapshot Format (Breaking Change)

The snapshot format has changed in 0.23.0 and is not backwards compatible with previous versions. The changes were specifically to help us support backwards compatibility and ease of upgrade in the future. Under the hood, Zeebe uses RocksDB to store aggregated state for each partition. Previously, values stored there were serialized using a mix of different approaches; this was now standardized to MessagePack, which should give us more flexibility in the future when it comes to schema changes.

Health check endpoint

Up until now, the Zeebe broker offered a single ready check HTTP endpoint (by default at http://zeebe-broker:9600/ready) which one could use, for example, to define a Kubernetes readiness probe. There is now an additional HTTP endpoint on the Zeebe broker (by default http://zeebe-broker:9600/health), which reports the overall health of a single broker (not to be confused with overall cluster health). If the broker is healthy, it will return an empty 204 response; if it is unhealthy, it will return a 503 error.

But what is the health of a broker? If a broker is healthy, then it means that it can process workflows, accepts commands, etc., perform all its expected tasks. If it is unhealthy, then it can mean three things: it is only temporarily unhealthy (e.g. due to environmental circumstances such as temporary I/O issues), it is partially unhealthy, or it is completely dead.

Under the hood, this was done by breaking the health of the broker as the combined health of its partitions. Each partition, in turn, monitors further components and reports their health. This essentially creates a supervision tree, allowing us to more easily pinpoint which components have failed. This is exciting, as although Zeebe can already tolerate crash failures, it currently does not automatically recover from partial failures. This lays the groundwork for major improvements in that aspect.

You can read more about this on the Zeebe documentation website.

OAuth2 client compliance

As part of 0.23.0, we’re happy to report that both the Go and Java official Zeebe clients are now fully OAuth2 compliant. Thanks to our user dbarentine for reporting the issue. You can read more about the issue and its fix for the Go client and the Java client on GitHub.

Google Cloud Stackdriver logging integration

For those deploying Zeebe on Google Cloud (such as our very own Camunda Cloud), Zeebe now offers the option to output its logs directly in a format that Stackdriver can ingest seamlessly. To enable the Stackdriver integration, you have to set the environment variable ZEEBE_LOG_APPENDER to Stackdriver. You can read more about the configuration in the Zeebe documentation.

What’s new in Operate 0.23

Note: Please note the new version number of Operate. With this release we
change the release version number to align with Zeebe. This leads to the
“strange” moment where you as a user upgrade from Operate 1.2 to Operate 0.23.
We think it is better for users to have the same version number for both
products and decided to do this step now instead of waiting longer.

Improvement handling instance operations

The new operations panel in the instances list view replaces the selections
panel. The old selections panel had some limitations in it’s usability and did not
give the user any visibility of the progress in case an operation was triggered.

With the new operations panel we remove the ability to create multiple
selections of instances and then execute operations on them, and focused more
on the visibility of running and finished operations.

For every operation trigger inside the application, i.e. cancel, retry or edit
variables, a new entry in the operations panel is created. The item reflects
the current progress of the operation, the creation and end time. The operation
also contains a link back to the affected instances, in case the user wants to
drill into or execute another operation on the same set of instances.

Highlighting of executed sequence flows

With the latest release of Operate the details view will now highlight the sequence flows which where taken in the process. This makes it easy to understand which path in the process was executed.

Automated state migration between versions

To allow users to migrate from one version of Operate to the next one we now ship migration scripts with every Operate release.

The migration uses Elasticsearch processors and pipelines to reindex data. It migrates data by executing following steps:

1. Remove old templates
2. Create new templates and indices
3. Create pipelines
4. Reindex old to new indices
5. Remove pipelines and old indices

That means you might need more disk space during the migration.

You can use the Operate docker container as init container and as main
container when running Operate in a Kubernetes environment like Google Cloud.

The init container needs just just another command:

['/bin/bash','/usr/local/operate/migration/migrate.sh','http://elasticsearch-host:9200']

The init container checks the version of stored data in Elasticsearch and
migrates it to the current version if necessary. If the new version of data
already exists, then no migration is performed and init container quits
successfully. Was the init container successful executed, then the application
Operate will be started (main container). Details about migration can be
found in the Operate deployment guide.

What’s new in Zeebe Modeler 0.9.0

Error End Event support

The latest release completes the support for error events by adding the new error end event to the list of supported elements. You can read more about error events in the Zeebe documentation.

Deploy diagram and start instances from modeler

To reduce the round trip time from modeling to deploying and executing a new workflow the modeler now has two new features.

When pressing the “Deploy current diagram” button a modal window will open, which allows you to enter the connection details of your Zeebe cluster.

After you successfully deployed your new model you can even start a new instance directly from the modeler by pressing the “Start Current Diagram” button.

Get In Touch

There are a number of ways to get in touch with the Zeebe community to ask questions and give us feedback.

• Join the Zeebe user forum
• Join the Zeebe Slack community
• Reach out to dev advocate Josh Wulf on Twitter
• Reach out to dev advocate Mauricio Salatino on Twitter

We hope to hear from you!