Versioning Process Definitions

For real life applications, it's crucial to understand how Camunda deals with evolving process definitions by means of versioning. As a rule of thumb, we recommend to version just the process, case and decision models, but not all other process application artifacts (like e.g. code classes or scripts, forms and other templates). Often you might not even want to run multiple model versions at the same time, then you have to think about migrate running process instances to new versions. When modeling very long running processes (> 6 months), consider cutting them into reasonable pieces to better manage your versioning requirements.
Versioning Process Definitions is also related to
  • Versioning
  • Version Migration
  • Long Running Processes
Versioning Process Definitions

Understanding Versioning

By default, deploying a process, case or decision definition means that the process engine will check if the version has changed. If it has, it will register that deployment as a new version of the definition. By default running instances will continue to run on the basis of the version they started with, new instances will by default be spawned from the latest version of that definition.

As a consequence, when looking directly at Camunda database tables you can see different versions in the process definition table and the running process instances which are linked to these versions:

database versions

Understanding Deployments

Camunda knows the notion of a deployment, containing a number of process (or case or decision) definitions. The platform versions all artifacts of a deployment in sync by default (see Configuration Property "isDeployChangedOnly" to change this behavior).

Using Semantic Versioning

Version numbers are automatically set by the Camunda engine. Hence you cannot influence the version number and chances are high that it differs on various staging systems (e.g. development, test, qa and prod).

Use Camunda’s camunda:versionTag attribute to tag your versions with semantically meaningful labels. You can use the semantic version to query the process definition you are interessted in:

ProcessDefinition processDefinition = repositoryService.createProcessDefinitionQuery()
        .versionTag("2.5.7") (1)
        .latestVersion()     (2)
        .singleResult();
1 Query only for the semantic version,
2 Include only the latest version of the process definition, in case you re-deployed multiple process definitions with the same semantic version, which is most likely to happen in local development environments.

We recommend to always set a semantic version to track the model seen during operations down to the right development tag in version control.

A nice approach is to use Maven Resource Filtering to let Maven set the verstionTag to the current Maven version.

Selecting the Best Versioning Approach

Running Versions In Parallel

You can run several versions of a specific executable model in parallel. The big advantage of that default behavior is that you can deploy changed process definitions without caring about running process instances. The process engine is able to manage running instances based on different process definitions in parallel. The disadvantage is, that one needs to deal with the operational complexity of different versions of the process running in parallel as well as the additional complexity in case those processes call sub processes which have different versions of their own.

Run versions in parallel for

  • development or test systems for which you do not care about old instances

  • phasing out existing instances as the existing instances need to finish based on the model they where created with, which often has legal reasons.

  • situations in which migration is not advisable, because it is too complex and too much effort when weighed against its upsides.

Migrating Process Instances to a New Version

Migrate running instances to the newest definition when

  • deploying patches or bug fixes of a process model.

  • avoiding operational complexity due to different versions running in production is a priority.

Camunda supports the nuts and bolts to do that by different approaches.

Using Process Instance Migration

Migrating process instances can be achieved either programmatically or by using Camunda Cockpit. Programmatically, you need to create a migration plan that describes how process instances are to be migrated from one process definition to another.

MigrationPlan migrationPlan = processEngine.getRuntimeService()
  .createMigrationPlan("exampleProcess:1", "exampleProcess:2")
    .mapActivities("assessCreditWorthiness", "assessCreditWorthiness")
    .mapActivities("validateAddress", "validatePostalAddress")
    .mapActivities("archiveApplication", "archiveApplication")
  .build();

You can then apply such a plan to a set of process instances selected by you.

Consult the manuals to learn more about doing Process Instance Migration programmatically.

Camunda Cockpit provides a powerful user interface to compose migration plans without writing code:

You can run your migration plan in batch mode and control the progress via Cockpit:

batch migration
Consult the manuals to learn more about Process Instance Migration with Camunda Cockpit.

An interesting option is, that you can export the migration plan you configured in Cockpit as JSON string. This migration plan can be applied later via REST-API (see Migration resource), making it possible to fully automate migration even if you do not want to programm a migration plan in Java.

Note that migration cannot (yet) change variables within the migration plan. If you have to adjust variables as part of your migration use an additional step using Process Instance Modification.

It’s important to understand that process instance migration - just as you would probably expect - maintains the full 'identity' of the migrated process instances including their unique IDs and their full history audit trail. However, as the process definition also might change fundamentally in between versions, this can have effects on the history log of a process instance which might be unexpected from an end user’s or operator’s perspective.

Using Process Instance Modification (Delete and Re-Start)

While process instance migration maintains the identity of process instances migrated to a new process definition version (see above), migrating process instance by means of process instance modification does not.

The Process Instance Modification API allows you to start instances of a new process definition version somewhere "in the middle" using the right set of process variables, like shown below. Hence you can delete the old process instance and start a new one in the right state with the same process variables.

ProcessInstance processInstance = runtimeService()
  .createProcessInstanceByKey("TwitterDemoProcess") (1)
  .startBeforeActivity("service_task_publish_on_twitter")
  .setVariables(variables)
.execute();
1 We create a process instance of our TwitterDemoProcess here, but do not start it regularly. Just before we publish our tweet on twitter. We also take care of the status of the process variables needed at that point by providing them as a map of variables.

As you see, with this tool you can also migrate running instances based on an old version of your process definition to a newer version.

With this approach, the full 'identity' of the migrated process instances is NOT maintained. This means that the migrated process instance is a new one and will just show a history from the point on at which it was started. Depending on your specific circumstances and use cases you might see that as an advantage over process instance migration (see above), too.

Avoid Versioning of Dependant Artifacts

When versioning process, case or decision definitions, one needs to be aware that the process of course communicates with the outside world, e.g. by calling services or by using forms to collect data input from human users. All the additional artifacts needed for that might depend on the details of each other in a subtle way.

Whenever possible, we recommend that you avoid to version other artifacts beyond the process, case and/or decision definitions, in other words, just version '.bpmn', '.cmmn' and '.dmn' files by using the default mechanism of the process engine. Embed all other artifacts (like e.g. classes, templates, scripts) into your "normal" deployment (e.g. a '.war' or '.ear' file) and don’t version them.

Of course, this approach requires that you manage the subtle differences needed by running process instances of old versions. There are various options to do that.

Some of the options discussed below might not sound 'ideal' from a theoretical point of view. However, in our experience they are absolutely good enough for real life purposes and much easier to understand than more complex approaches. We consider understandability by every team member as a very important argument!

The following options refer to this example of artifacts in your process application:

other artifacts

Option 1: Keep the Artifact Backwards Compatible

Extend the functionality of e.g. a method in MyClass.java in a way which can still deal with "old" process instances.

public class MyClass {
  public void doSomething(Long customerId) {
        if(customerId != null) { (1)
          // new code introduced
    }
  }
}
1 Assume you introduced a customerId in the new version of the process. Your code can still deal with old cases not aware of a customerId.

Option 2: Introduce a Switch for Different Versions

Extend the functionality of e.g. task-form.xhtml in a way which can still deal with an old process definition.

<c:choose>
    <c:when test="#{processInstance.hasVariable('gender')}"> (1)
        <p>Gender: #{processInstance.getVariable('gender')}</p>
    </c:when>
    <c:otherwise>
       <!-- just if needed -->
    </c:otherwise>
</c:choose>
1 Assume your old process definition did not know the process variable gender, but the new version holds it and you need to display its content. Even though you know that it must exist in the new version, your check for the existence of the variable serves as a switch for different process definition versions.

Option 3: Introduce a New Artifact For Different Versions

Change the functionality of e.g. task-form.xhtml, by adding a new version to the application and referencing it from the new version of the process definition, while the old version will continue to use the first version of the form. Of course, this approach would be applicable for other artifacts, too, as e.g. MyClass.java

other artifacts versioned
Often it is preferable to manage different versions by means of folders/packages. Just make sure to have a clear and straightforward convention to keep track of the versions.

Option 4: Use a Deployment independent and Unversioned External Resource

Use an unversioned and deployment independent external resource for your process definition(s) by adding the external resource to the container. In this scenario any process can reference the external resource as long as the process is part of the same container as per the image below. This approach, for example, can be applicable to scripts or Java resources.

external resource
Often it is preferable to manage different versions by means of folders/packages. Just make sure to have a clear and straightforward convention to keep track of the versions.

Dealing with Long Running Processes

In general, do not be concerned with deploying long running processes which might run days, weeks or even months. After all, this is exactly what Camunda was built to properly deal with.

Having said that, also review the possibilities the process engine provides with respect to cutting process definitions (e.g. via message exchange or via call activities) and migrating running process instances. But even though it’s possible to migrate running process instances to a new version (see below), it’s typically a bit of effort. Therefore the information presented in the following sections is meant to enable your conscious decision at which points it might make sense for you to avoid the necessity for migration by cutting processes and which aspects of versioning behavior you can control by doing that.

Cutting Very Long Running Processes into Pieces

The longer the lifespans of process instances are, the bigger the risks that you might want to exchange important software components like e.g. the process engine itself. Typically, very long running end-to-end processes (running longer than six months) have periods without activity (e.g. waiting for a certain date in the future). Cut the process into several independent process definitions at these points.

1 After the mobile phone was shipped, we finish the first process instance and just keep a reminder for the renewal in 24 months.
2 We periodically check due renewals and start new process instances whenever necessary.
We typically don’t model such processes in one diagram it’s shown here as a way to show the message flow. Typically, we would rather use a separate diagram per executable process and either leave out the other process completely or show it as a collapsed pool.

Also try to avoid modeling the complete life-cycle of very long living objects, like a life insurance contract. Only capture the active phases as separate processes (e.g. "Policy Issuing", "Address Change", "Cancellation" or "Death").

Having said this, we want to emphasis that the engine is perfectly fine with handling lots of process instances for a long time. So if you want to have process instances waiting for months or years, you can still do so. After all, Camunda provides a proper API for version migration and other tools. Just make sure you think about all resulting implications.

Using Call Activities to Influence Versioning Behaviour of Pieces

When calling separately modeled sub processes (i.e. Call Activities), the default behavior of the process engine is to call the latest deployed version of that sub process. You can change this default 'binding' behavior to call a specific version or the version which was deployed together with the parent process. Learn more about Call Activities in the User Guide.

Keeping in mind pros and cons of versioning as discussed above, we can therefore encapsulate parts of a process, for which we want to be able to change the runtime behavior more often into such call activities. This is an especially useful consideration for long running processes.

1 We could decide that we always want to follow the latest shipping process changes, even if the rules for shipping changed while we are in the order acceptance phase. We for example reason that this acceptance phase could sometimes take a long time, because the procurement for goods currently not shelved happens within that phase.
2 Contrary to that, we could decide that the order billing always happens according to the rules valid at the moment we received the order and instantiated the parent process (deployment). We for example reason here that it is critical that the billing follows the rules communicated to the customer together with the offer.

Bonus  Versioning Dependant Artifacts

Sometimes you cannot avoid to version dependant artifacts as recommended in this Best Practice. In this case the following two options can be used.

Approach 1: Use the BPM Deployment

This more complex approach is just recommended if you are sure you really need it: use the deployment versioning mechanism of Camunda to version other interpreted artifacts (e.g. scripts, templates, html forms), too. In principle, you can deploy and version as many other artifacts as you like together with your process definition, e.g. by declaring them as a resource in the process archive configuration.

bpm deployment
Please don’t do this for compiled java classes!

At runtime, you will need to look up the needed artifact from your deployment by using the API:

byte[] getForm(String taskId) { (1)
    Task task = taskService.createTaskQuery().taskId(taskId).initializeFormKeys().singleResult(); (2)
    ProcessDefinition processDefinition = repositoryService.createProcessDefinitionQuery()
        .processDefinitionId(task.processDefinitionId).singleResult(); (3)
    try {
        return repositoryService.getResourceAsStream(
            processDefinition.getDeploymentId(), task.getFormKey() (4)
        ).getBytes();
    } catch (DeploymentResourceNotFoundException e) {
        // TODO form not found!
    }
}
1 Assuming you want to retrieve the content of an HTML file contained in your process application.
2 Query the user task to get the process definition id.
3 Query the process definition as we need the deployment id from it to query resources.
4 With the deployment id we can now query arbitrary resources by name (in this case the name is set via the form key).

Approach 2: Use the Java Container Deployment

This even more complex approach is just recommended if you believe you cannot do without it: use the deployment mechanism of your Java container to deploy a versioned deployment file (like e.g. another version of a '.war' or '.ear' file). This way, you can manage different versions of compiled Java classes, too.

container deployment

As you see in the picture above, you now need to take care of explicitly versioning the process definition, either with a versioned process definition 'id' as shown above, or with the mentioned semantic version which allows you to select the correct version from the code of your process application.

This approach has the advantage that the identity of your process versions still remains intact from point of view of other applications (like e.g. Camunda cockpit). At startup time of your process application, you need to register the correct version at the Camunda platform yourself (as the default mechanism register a Java deployment for either all process definitions or just the latest version of it):

@ProcessApplication
public class MyProcessAppliaction extends EjbProcessApplication {

    @PostDeploy
    public void registerProcessApplication(
        ProcessEngine processEngine,
        ProcessApplicationInfo processApplicationInfo
    ) {
        // Find Process Version based on XML attribute
        // …
        // Register Process Archive for Process Version
        processEngine.getManagementService()
            .registerProcessApplication(
                 processDefinitionDeploymentId,
                 processApplicationReference);
    }

}
Please be aware, that the operational complexity of this third approach is especially high, hence we seldom meet customers using parallel Java deployments in reality. Keep in mind that external resources like service endpoints (urls) eventually need to be versioned, too. In most cases it is preferable to simply copy classes into separate packages like described in approach 1.

Bonus  Beyond, but part of, versioning process definitions

Once the process and it’s resources are ready to be deployed it is important to ensure continuous integration. In other words it is necessary to update the deployed resources whenever there is a new version or resource. It is possible to do this manually, however most Companies nowadays use a continuous delivery tool. It is also recommended to have the process application,Maven project and files in a repository like Github or Bitbucket and then use Jenkins to pull from the repository and deploy the artifacts to the production environment.

continous delivery

No guarantee - The statements made in this publication are recommendations based on the practical experience of the authors. They are not part of Camunda’s official product documentation. Camunda cannot accept any responsibility for the accuracy or timeliness of the statements made. If examples of source code are shown, a total absence of errors in the provided source code cannot be guaranteed. Liability for any damage resulting from the application of the recommendations presented here, is excluded.

Copyright © Camunda Services GmbH - All rights reserved. The disclosure of the information presented here is only permitted with written consent of Camunda Services GmbH.