Last Wednesday, July 17, we announced the first production ready release of Zeebe, Camunda’s new cloud-native workflow engine for microservices orchestration. Zeebe is a new code base, written from scratch, putting forward a completely new way of architecting workflow engines for microservices and distributed systems.
This blog post is a quick overview of why and how we did that.
Why we created Zeebe
About 6 years ago, Camunda entered the workflow space with a new approach to BPM. While our approach was very successful, it was foreseeable that something big was coming that would transform the tech landscape: microservices and distributed systems.
To meet an ever-growing need to define new business models and disrupt existing ones, companies must scale software development. If software is eating the world, then microservices and distributed systems are the forks and knives of that digital transformation.
It was clear to me that this migration to microservices architectures could only succeed if we figured out the right way of implementing workflows and business processes in this new distributed reality. However, this was such a radical change that it was not going to be sufficient to simply re-apply existing solutions to new problems. Something fundamentally new was needed and that came to be Zeebe.
(For a more in-depth overview, Read Mike’s post: Workflows Are Everywhere.)
How Zeebe is different
Zeebe puts one thing front and center, and then everything else naturally follows from there: Events!
In distributed microservices architectures, events are the new first class citizens. Events make it possible to record and publish state changes in one domain (microservice) such that other domains (microservices) can subscribe and react to them. This simple yet powerful principle decouples microservices from each other and introduces scalability and reproducibility. However, a problem that was not yet solved well is how to coordinate many (tens to hundreds) of actions implemented by different microservices to achieve a specific outcome. Or in other words: how to implement complex workflows. As the complexity grows, it becomes increasingly important to have visibility into such flows
And Zeebe provides exactly this – the ability to execute, manage, and monitor workflows that span across microservices and need to be scaled massively in the cloud.
Zeebe interprets workflow on top of event streams. Event streams flow into Zeebe (it is very easy to subscribe to external events from a workflow) and event streams flow out of Zeebe (as it processes workflows, it exports a complete event log into external systems). Given this event-driven way of interacting with the external world, it is only logical that Zeebe builds on top of stream processing internally, instead of a database. This simple fact is what makes Zeebe unique and completely different from any other workflow engine out there that we are aware of. It is at the core of Zeebe’s cloud native architecture, enabling it to scale horizontally, be fault tolerant through replication, and integrate natively into microservices architectures.
It only took me about 2 years to figure out how to do that, and then it only took some of the smartest people I know 2 more years to build it for real. And now, it is finally here! : )
If this has piqued your interest, you can read about the architecture in much more detail in Bernd’s excellent posts:
