The upcoming 0.22 release of Zeebe includes a feature many users have been asking for: the ability to start a workflow and retrieve its outcome with a single command.

The new gRPC command CreateWorkflowInstanceWithResult is available for testing in the current SNAPSHOT Docker image of Zeebe and the zeebe-node-next version of the Node.js client.

This command starts a workflow instance and returns the outcome when the workflow completes.

Use-cases

A common scenario is to start a workflow in response to a REST request, and send back the outcome from the workflow in the REST response. Previous implementations of this relied on a polling worker to retrieve the outcome, and the use of subscriptions to correlate it with the REST request context (See the “Zeebe Workflows Inside a REST Request/Response” blog post and the zeebe-node-affinity package). This scenario makes sense when the workflow is short-running.

A Tour of Related Issues and PRs

The initial feature request for awaitable workflow outcomes – #2896 – got a lot of traction and input from the community. This is the best way to get something implemented in Zeebe: open a feature request, make a compelling use-case with a good description, and get wider community support for it.

The work to implement the functionality in the broker has been done by Deepthi Akkoorath, and is in this pull request. If you work with Zeebe and are interested in this feature, I recommend you read through the changes. I can’t overstate the value of reading the source code to build up your understanding of using Zeebe. All the answers are in there. You may not understand everything, but you never will if you don’t start somewhere, so dive in!

Awaiting workflow instance results is implemented in an upcoming release of the Java and Go client (#3286), and in the testing version of the Node.js client, zeebe-node-next (example).

The work to support it in zbctl is still to be done, and is being tracked here.

An example using Node.js

Here is how you use the feature with the current SNAPSHOT Zeebe server and zeebe-node-next:

import {} from 'zeebe-node-next'

async function main() {
    const zbc = new ZBClient()
    await zbc.deployWorkflow('./test.bpmn)

    const initialVariables = {
        clientId: 1
    }

    const outcome = await zbc.CreateWorkflowInstanceWithResult(
        'test-process-id',
        initialVariables,
    )

    console.log(outcome.variables)
}

main()

Dealing with gRPC request timeout

An important consideration here is the gRPC request timeout. By default, this is the request timeout configured on the Zeebe gateway (15 seconds, out of the box). If your gRPC request times out before the workflow completes, the client will receive a request timeout error back from the gateway. If this happens, you will not have a workflow instance Id for the workflow that you just created.

You can retry the request – note that each retry creates a new workflow instance, so you should ensure that any side-effects in the workflow are idempotent. If the broker timeout is due to load, it may succeed on a subsequent execution. But if the workflow takes intrinsically longer than the timeout, you will never get a result back. So you need to configure the request timeout to be of sufficient length.

You can override the gateway request timeout by using a different signature for the createWorkflowInstanceWithResult call in the Node client (the C#, Go, and Java clients use a builder pattern, so constructing the command differs in those clients):

async function main() {
    const result = await zbc.createWorkflowInstanceWithResult({
        bpmnProcessId: processId,
        variables: {
            sourceValue: 5,
            otherValue: 'rome',
        },
        requestTimeout: 25000,
    })
    return result
}

main()

Also to note: since no workflow id is returned until the workflow completes, if your system experiences a failure while the request is pending (either client-side or on the broker), you will have no way to correlate the request.

Constraining the returned variables

A further feature (#3253), that hasn’t been merged at the time I wrote this blog post, is the ability to constrain the variables returned by the call.

When this lands, you’ll be able to do this:

async function main() {
    const result = await zbc.createWorkflowInstanceWithResult({
        bpmnProcessId: processId,
        variables: {
            sourceValue: 5,
            otherValue: 'rome',
        },
        fetchVariables: ['finalBalance']
        requestTimeout: 25000,
    })
    return result.variables.finalBalance
}

main().then(console.log)

Summary

The new CreateWorkflowInstanceWithResult command allows you to “synchronously” execute workflows and receive the outcome. It’s a popular feature request in the community and will reduce the complexity of Zeebe systems in production that have been using complex patterns to achieve this functionality.

The feature is ideally suited to workflows that retrieve information in response to a request, and return it in the response. If your workflow mutates system state, or further operations rely on the workflow outcome response to the client, take care to consider and design your system for failure states and retries.

Bonus

I live stream coding with Zeebe and Node.js, weekly on Twitch.

Here are a couple of videos from this week’s stream, implementing support for this feature in the Node client.


  • Camunda Platform 8.1 Released – What’s New

    We’re extremely excited to announce the release of Camunda Platform 8.1.  In this post, we will go into the highlights of this release including Web Modeler, Connectors, Zeebe, Operate, Optimize, and more. Here’s the highlights of what’s included in this release: Easier Installation and maintenance for Camunda Platform Self-managed Hot backups for Self-Managed Token-Based authentication Modeler updates Process Templates FEEL Expression Editor Inline Annotations Changes to Camunda forms Connectors updates Workflow engine updates Operate updates Optimize updates Let’s get started! Easier installation and maintenance for Camunda Platform Self-Managed Improved Guides and Support for OpenShift and Amazon EKS (AWS) With this release, we are introducing official support for deploying and running Camunda Platform 8 Self-Managed on OpenShift and Amazon EKS (AWS)....

    Read more
  • Camunda Platform 7.18 Released – What’s New

    We’re extremely excited to announce the release of Camunda Platform 7.18 Here are some of the highlights: Cockpit: Improved instance selection to avoid unintended batch operations Cockpit: Easier navigation of high batch operation volumes MDC logging features Camunda Forms in Tasklist: Populate select options via process data Exception error codes Filter and order tasks by last updated WebSphere Liberty New and retired supported environments You can download Camunda Platform or run it with Docker. For a complete list of the changes, please check out the release notes. For patched security vulnerabilities, see our security notices. If you want to dig deeper, check out the source code on GitHub. Cockpit: Improved instance selection to avoid unintended batch operations Previously, when performing...

    Read more
  • Increase your resilience with new regions in...

    We’re excited to announce new regions for Camunda Platform 8 SaaS that further strengthen the resilience, performance, and data requirements for our SaaS customers. Enterprise customers can now choose to create their clusters in a number of regions including Europe West, US Central, US East, and the most recent addition, Australia South East. This provides multiple benefits, such as increasing availability, allowing to set up fail-safes, adhering to regulatory requirements for regional data storage, and reduced latency due to closer physical proximity. Resilience and redundancy in technology are important for all modern organizations and are top of mind for many. Across industries, organizations need a solution that scales effectively and is resilient to provide a good base for process orchestration....

    Read more

Ready to get started?

Still have questions?