The upcoming 0.22 release of Zeebe includes a feature many users have been asking for: the ability to start a workflow and retrieve its outcome with a single command.

The new gRPC command CreateWorkflowInstanceWithResult is available for testing in the current SNAPSHOT Docker image of Zeebe and the zeebe-node-next version of the Node.js client.

This command starts a workflow instance and returns the outcome when the workflow completes.

Use-cases

A common scenario is to start a workflow in response to a REST request, and send back the outcome from the workflow in the REST response. Previous implementations of this relied on a polling worker to retrieve the outcome, and the use of subscriptions to correlate it with the REST request context (See the “Zeebe Workflows Inside a REST Request/Response” blog post and the zeebe-node-affinity package). This scenario makes sense when the workflow is short-running.

A Tour of Related Issues and PRs

The initial feature request for awaitable workflow outcomes – #2896 – got a lot of traction and input from the community. This is the best way to get something implemented in Zeebe: open a feature request, make a compelling use-case with a good description, and get wider community support for it.

The work to implement the functionality in the broker has been done by Deepthi Akkoorath, and is in this pull request. If you work with Zeebe and are interested in this feature, I recommend you read through the changes. I can’t overstate the value of reading the source code to build up your understanding of using Zeebe. All the answers are in there. You may not understand everything, but you never will if you don’t start somewhere, so dive in!

Awaiting workflow instance results is implemented in an upcoming release of the Java and Go client (#3286), and in the testing version of the Node.js client, zeebe-node-next (example).

The work to support it in zbctl is still to be done, and is being tracked here.

An example using Node.js

Here is how you use the feature with the current SNAPSHOT Zeebe server and zeebe-node-next:

import {} from 'zeebe-node-next'

async function main() {
    const zbc = new ZBClient()
    await zbc.deployWorkflow('./test.bpmn)

    const initialVariables = {
        clientId: 1
    }

    const outcome = await zbc.CreateWorkflowInstanceWithResult(
        'test-process-id',
        initialVariables,
    )

    console.log(outcome.variables)
}

main()

Dealing with gRPC request timeout

An important consideration here is the gRPC request timeout. By default, this is the request timeout configured on the Zeebe gateway (15 seconds, out of the box). If your gRPC request times out before the workflow completes, the client will receive a request timeout error back from the gateway. If this happens, you will not have a workflow instance Id for the workflow that you just created.

You can retry the request – note that each retry creates a new workflow instance, so you should ensure that any side-effects in the workflow are idempotent. If the broker timeout is due to load, it may succeed on a subsequent execution. But if the workflow takes intrinsically longer than the timeout, you will never get a result back. So you need to configure the request timeout to be of sufficient length.

You can override the gateway request timeout by using a different signature for the createWorkflowInstanceWithResult call in the Node client (the C#, Go, and Java clients use a builder pattern, so constructing the command differs in those clients):

async function main() {
    const result = await zbc.createWorkflowInstanceWithResult({
        bpmnProcessId: processId,
        variables: {
            sourceValue: 5,
            otherValue: 'rome',
        },
        requestTimeout: 25000,
    })
    return result
}

main()

Also to note: since no workflow id is returned until the workflow completes, if your system experiences a failure while the request is pending (either client-side or on the broker), you will have no way to correlate the request.

Constraining the returned variables

A further feature (#3253), that hasn’t been merged at the time I wrote this blog post, is the ability to constrain the variables returned by the call.

When this lands, you’ll be able to do this:

async function main() {
    const result = await zbc.createWorkflowInstanceWithResult({
        bpmnProcessId: processId,
        variables: {
            sourceValue: 5,
            otherValue: 'rome',
        },
        fetchVariables: ['finalBalance']
        requestTimeout: 25000,
    })
    return result.variables.finalBalance
}

main().then(console.log)

Summary

The new CreateWorkflowInstanceWithResult command allows you to “synchronously” execute workflows and receive the outcome. It’s a popular feature request in the community and will reduce the complexity of Zeebe systems in production that have been using complex patterns to achieve this functionality.

The feature is ideally suited to workflows that retrieve information in response to a request, and return it in the response. If your workflow mutates system state, or further operations rely on the workflow outcome response to the client, take care to consider and design your system for failure states and retries.

Bonus

I live stream coding with Zeebe and Node.js, weekly on Twitch.

Here are a couple of videos from this week’s stream, implementing support for this feature in the Node client.


  • Title slide that reads "Why Camunda 8"

    Why R-KOM Chose Camunda Platform 8

    In this blog series, we highlight the customers who have chosen to utilize Camunda Platform 8 and explore the challenges those companies are attempting to overcome using process orchestration. For the latest installment of Why Camunda 8, we spoke with R-KOM, a telecommunications company based in Regensburg, Germany. When R-KOM was founded in 1997, its shareholders pooled their telecommunications infrastructure, which had evolved over decades with utility networks for water, electricity, and gas. Initially, R-KOM’s services were limited to business and the public sector, but now it has developed further in line with demand. Over the years, the company’s high-performance infrastructure and a broad range of products have grown. Today, R-KOM has a number of city networks in Eastern Bavaria...

    Read more
  • Title slide that reads "Why Camunda 8"

    Why Gruner + Jahr Chose Camunda 8

    In this new blog series, we explore the reasons why customers are migrating to Camunda 8. For our first installment of Why Camunda 8, we talked to Gruner + Jahr, one of the largest premium magazine publishers in Europe. G+J includes such established (German) print and digital brands as STERN, GEO, BRIGITTE, ESSEN & TRINKEN, and SCHÖNER WOHNEN, as well as younger brands such as CHEFKOCH, BARBARA, BEEF, 11FREUNDE. In addition to the numerous print and digital media offerings, G+J offers products and licenses such as the SCHÖNER WOHNEN collection. The digital business contributes more than half of revenues and is exhibiting continued strong growth. Indeed, the company’s digital products lead the rankings in all publishing segments, from news through...

    Read more
  • Camunda Platform 8.1.0-alpha3 Released

    We’re excited to announce the release of Camunda Platform 8.1.0-alpha3. If you’d like to get started with Camunda Platform 8.1.0-alpha3 right away, you can sign up for a free trial now. Create Process Instance Starting at User-Defined Elements An often requested feature is now available as a preview with Camunda Platform 8.1.0-alpha3: create a process instance starting at user-defined elements. When creating a process instance through the CreateProcessInstance RPC, the process instance is started at the default none start event. For testing purposes, you may want to start at one (or multiple) of the other elements. This feature is now available through both the CreateProcessInstance RPC and the CreateProcessInstanceWithResult RPC. It is available for use in the Zeebe Java client...

    Read more

Ready to get started?

Still have questions?