How to Create a Central Process Automation Platform
18 min read
Scale your process automation efforts with a centralized approach
The question of whether to centralize process automation doesn’t always have a simple answer. There’s a continuum of possibilities when it comes to managing process automation, all of which are contingent upon how your organization and employees prefer to operate.
Instead, it’s better to look at automation on a spectrum. On one end, each team uses its own technologies, platforms and sets of rules. On the opposite end, a central automation platform serves as a hub for all process automation efforts, and a single team provides the necessary resources for stakeholders upon request. In between these polar opposites, a variety of approaches exist. For example, some companies may choose to implement a Center of Excellence (CoE) that serves as a consultancy for the rest of the organization, and is composed of enterprise architects and developers.
With those nuances in mind, it’s important to note that centralizing process automation looks a lot different today than it did in the days of legacy Business Process Management Systems (BPMS). With the rise of SaaS and cloud-based offerings, a CoE can select a central automation platform, yet allow teams the flexibility to provision their own resources and deploy their own self-service process models. Instead of enforcing strict rules, this model can provide a level of governance that’s necessary within many larger organizations.
Why Centralize Process Automation?
Given all of the potential options for organizations, why centralize process automation? It’s important not to force the idea of a central platform on teams that are successful at selecting and deploying their own tools independently. In some cases, these decentralized approaches work perfectly well and shouldn’t be disrupted.
However, in cases where it’s not as clear-cut, a central automation platform can take the burden off of busy developers to evaluate workflow engines, negotiate with vendors, and continuously maintain an extensive software stack throughout the software development lifecycle. Rather than contending with these decisions, developers can simply work within a SaaS-based automation platform to create and deploy their own process models. This can scale much faster than a decentralized approach, and ease the burdens of long-term maintenance.
For many teams, the governance and visibility of a central automation platform can help processes operate efficiently and effectively. Teams don’t need to reinvent the wheel whenever another process needs to be automated. They can simply spin one up, without worrying about compliance, costs, or other potential risks. And, with the addition of a CoE, developers can get the guidance they need to operate the platform successfully. Often, that means they don’t need specialized expertise in a particular technology, or they can receive the coaching and training they need from the CoE. Finally, the CoE can see where processes are operating effectively, and implement changes where response times are sub-optimal.
If centralizing process automation sounds appealing to you, read on. We’ll walk through an approach that many teams have successfully used to kick off a larger, organization-wide process automation effort.
A step-by-step approach to centralized process automation
Depending on the nature of your team, the steps in this process may be in a different sequential order. However, many successful organizations start with a proof-of-concept (PoC) and lighthouse project to determine whether a central process automation platform will be effective for their organization. From there, it’s much simpler to prove the platform’s value and roll it out to a variety of stakeholders.
Let’s walk through one way this process could work.
Step 1: Create a PoC and lighthouse project
Start first with a pilot project. The goal of this project is to define and validate both the architecture and technology stack. Very often, this pilot project is set up as a proof of concept (POC). However, it is important to go-live with that pilot to learn about all aspects of the process automation solution throughout the full software development life cycle. You should choose a scenario where you can show at least some of the benefits of workflow automation (e.g., increased efficiency, effectiveness, compliance). Many people, including decision-makers, will be interested in quantifiable results.
Learn more: Doing a proper PoC
A team of two to four people is optimal for doing a PoC. These team members should have the skills required to use a standards-based process modeling language such as Business Process Model and Notation (BPMN), execute the project technically in your environment, and present the results to a group of business stakeholders. In most cases, it makes sense to mix business and IT stakeholders in this team. First, this team should focus on defining the goals for your PoC, and a scope for the project that’s relevant to your business.
Defining PoC goals and scope
Before planning and carrying out a PoC, you should consciously clarify the specific goals you want to achieve. Typical goals might include:
- Verifying that a specific approach or tool works within your scenario
- Showcasing an example that gets internal stakeholders to buy into a larger automation effort
- Working through granular questions before kicking off a larger effort.
From there, the PoC team should select a single process, use-case, or decision to automate to help accomplish your goals. Typically, this process should be relevant to core business stakeholders, clearly demonstrate the ROI of automation, and be feasible within a relatively short window of time.
It’s easy to make the mistake of choosing an overly ambitious project as a PoC, but keeping the scope of work reasonable is critical to success. Another common mistake is focusing on too many goals at once. An example might be trying to get a technical prototype running in your target architecture, while getting lost in the finer details of the user interface. It’s important to make up your mind on a single goal. There’s always an opportunity to do multiple PoCs to account for different goals.
Kicking off the PoC and presenting results
An ideal PoC team should include a moderator to keep the effort on track, a developer or enterprise architect to implement the PoC in practice, and an analyst to present the use case and business requirements to a wide variety of stakeholders. The process of running a PoC can last anywhere from a few days to a few weeks, but should not extend for too long beyond that time window.
Once the project is complete, the ideal scenario is for the PoC team to present the results in a workshop format. Key stakeholders who are concerned about process automation – from technical leaders to line of business leaders – should be present to ask questions about the feasibility of the project, next steps, and ways to implement a similar approach with future automation efforts.
Key questions to ask might include:
- Can we integrate an orchestration engine into our architecture?
- Does the development approach fit into our organization’s approaches, or how can we adjust?
- How can we model a specific business domain problem using BPMN and/or DMN??
- What knowledge is needed for the business and development teams to kick this off at scale? Do we need to hire, or can we use the skills in-house?
- How much effort will typically be needed for these kinds of projects?
- What are the impacts of process applications on operations?
- Can the orchestration platform handle the scale we need?
Soon after running a successful pilot, you should tackle a lighthouse project. This project has a broader, more realistic scope and can be better leveraged to show off architecture, tooling and value of workflow automation to other people and teams within your organization. The lighthouse project can take all of the learnings of the PoC into account, and refine some of the initial PoC code as needed. For this project, select a relevant use case that will deliver business value with your process automation project right from the start.
Overall, starting with a project rather than a program is important because:
- It enables you to focus on a concrete use case and avoid strategic platform initiatives for as long as possible. Doing too much strategic work too early has a high risk that the team won’t deliver any business value for a long time. The chances of getting stuck in shaping a complex platform are much higher without first understanding concrete use cases.
- Agile development approaches can help your team develop workflow solutions iteratively and incrementally. This allows you to learn fast and let these learnings correct your course.
Step 2: Assemble your Automation Center of Excellence
Depending on your organization, you may choose to assemble a CoE to manage the rollout of your process automation solution to the rest of the organization. Some teams may choose to assemble an automation CoE before embarking on a pilot and lighthouse project. Others may formalize a CoE after these projects have been successfully completed. This order of operations depends largely on the culture of your team and how much structure you’d like to implement up front.
Additional Guide: Building a Superior Automation Center of Excellence
Either way, a CoE team will most likely be responsible for:
- Assessing current process automation solutions in the organization
- Evaluating process automation tools
- Implementing the process solution in a pilot/lighthouse project, or evaluating the results of a pilot/lighthouse project
- Providing the training and guidance needed to empower developers to model their own workflows (e.g. using BPMN and/or DMN standards)
- Assisting the rest of the organization with using the automation platform for their own use-cases.
In other words, the CoE’s job is to evaluate process automation technology and to help decide the right tool for the job at hand. Typically, these CoEs also manage technologies around robotic process automation (RPA) or skill-based routing for human tasks.
The CoE also creates and maintains internal best practices. Many process automation and orchestration platforms like Camunda offer best practices documentation as a basis from which to start. From there, you should further document decisions, constraints or additions that apply to your company.
For example, one large bank developed a “self-service portal” within the CoE over the course of two years. This portal contains getting started guides, project templates and some reusable components as maintained libraries. This setup allowed most projects to get going on their own, including projects staffed by offshore IT integrators.
The CoE can also foster a community, simply by being available to talk to. Additionally, you can create a forum, a Slack channel or regular face-to-face or web meetings to encourage knowledge-sharing. The right choices for this collaboration depends heavily on your company’s culture.
It is also worth investing in internal marketing, as it is important that other projects know about the CoE and automation projects. You might even want to talk publicly about your use case to help other organizations in a similar situation understand how to successfully implement an automation solution.
Tips for the CoE on Rolling Out the Central Automation Platform
- Start with a project, not a program.
- Don’t start big, strategic endeavors too early in your journey. Instead, go step-by-step until you are ready to scale.
- Resist the temptation to create your own bespoke platform when highly customizable alternatives are available.
- Get buy-in from your decision-maker. This is much easier to obtain when there is some real pain that your workflow is going to solve.
- Let your lessons learned influence your target picture. Don’t just adopt a consulting company’s best practices without testing first.
- Make sure to give experienced people the opportunity to help with follow-on projects.
- Capture best practices and ensure knowledge sharing.
- Provide reusable components or prebuilt connectors if they increase productivity.
- Establish an internal consulting approach, most likely organized as a CoE. Identify and nurture at least one well-known champion in the enterprise that can drive the topic.
- Define learning paths for new people or teams.
- Make sure to let automation project teams breathe and make their own decisions.
Step 3: Expand the Usage of your Process Automation Platform
Depending on the culture in your organization, expanding the usage of process automation may look different. As mentioned above, process automation efforts typically operate on a spectrum, based on the level of governance and control required at your organization.
In some cases, a CoE may simply make others in the organization aware of an existing project as an example they may use for inspiration. For a bottom-up adoption journey, this motion is powerful. If a CoE team has great success with a project and tells that story internally, it can sometimes serve as enough of a motivator for others to pursue an automation project with the same technology. However, in this model, some developers may choose their own solutions.
While this approach offers the most flexibility and freedom, it can sometimes be risky to let every team choose the tools they want for automation. The selection process can quickly become backed by trends and personal preferences, rather than data on what works best. Also, it can waste the developers’ time in vetting separate tools for each use case. It’s important for everybody to understand that certain technology decisions are a commitment for years, and sometimes even decades. These decisions and the resulting maintenance affect more than just the current team.
In cases where more governance is required, a process automation rollout may look like using the same platform across the entire organization.
One scenario could resemble a set of distributed processes managed by individual teams on an internal cloud server, or hosted as a managed service by your automation platform provider.
In other cases, this might mean using the same platform to manage all automation efforts centrally, using resources created by the CoE. These resources could include training, best-practices guides, project examples, and in some cases, templates.
Yet another common approach is to use the CoE as an architecture board that defines project automation guardrails. Ideally, this board does not dictate arbitrary standards but maintains a list of approved tools and frameworks. Whenever a team wants to use something that is not (yet) on the list, they have to discuss it with the board. These discussions help teams learn about alternatives that are better suited, or unveil important questions around maintenance. But of course, convincing arguments can get a green light.
The goal of leveraging a central automation platform should not be to impose unnecessary restrictions on your team. Instead, it should be to empower them to kick off their own process automation efforts as quickly and efficiently as possible. Ideally, the CoE never becomes a bottleneck to progress.
Case Study: Goldman Sachs and Camunda
Since 2015, Goldman Sachs has used Camunda Platform as the engine of its microservices orchestration platform to deliver automation at enterprise scale. With more than 60,000 unique users working with the platform each year, Goldman Sachs sees about six million tasks executed on its microservices automation platform each week.
In 2020, the company’s priorities for the microservices orchestration platform changed. As increasing demands on the service emphasized the need for throughput, scalability, and resilience, Goldman Sachs sought to build a new platform capable of adapting to these use cases. At the same time, the internal payments team at the company recognized a need for a platform that would provide additional security, the ability to quickly recover from disruptive incidents without data loss, and an overall smoother payment handling process.
Watch the video: Enabling Core Banking Use Cases with Camunda Platform 8
Recognizing the need for a solution that could provide microservice orchestration across all of its teams and meet the changing demands of its clients, Goldman Sachs set out to build a new platform. Goldman Sachs’ Enterprise Process Automation Platform was born.
Why Build a Central Automation Platform?
The vision for the Enterprise Process Automation Platform was to centralize all required operations to provision, model, deploy, execute, and monitor any workflow-enabled application. Building this platform with Camunda Platform 8 as the horizontally scalable BPMN workflow engine would enable a higher level of automation, functionality, and adaptability for both Goldman Sachs and its clients.
Some of the key features of this new platform included:
- Automated data retention, data purging, data backups, and data lake integration to enable analytics.
- Cluster provisioning integrated with data forms for better load balancing.
- Alert automation for self-protection and self-healing.
- Mirrored user experience for Camunda Platform 7 and Camunda Platform 8 users for consistency.
- Integration with manual tasks and forms using current Goldman Sachs proprietary tooling to improve process automation.
Finally, it was essential to Goldman Sachs that this new platform offered maximum visibility.
“When the platform is running, we are provisioning a lot of information that is important to put together. We expose this information in the workflow control center and separate this information into separate categories for the platform provider and clients,” said Javier Sabino, global architecture lead, digitization and workflow engineering, Goldman Sachs.
Extending Camunda with Custom Extensions
Goldman Sachs also extended Camunda Platform 8 to incorporate non-functional requirements for five key categories. Sabino explains:
- Performance Testing: “Clients come to us with non-functional requirements before they even have their services ready. So we invested in a testing harness which allows us to configure clusters, execute services, and produce some reports which will show the best configurations for clusters to serve clients’ purposes.”
- Security: “We integrated Camunda with our single sign-on solution…we also added extensions to support data encryptions for clients.”
- Disaster Recovery: “We invested in a solution that helps Goldman Sachs recover from disasters, such as losing full regions, in less than fifteen minutes.”
- Observability: “We have solutions which show you how your workflows are progressing. But we also invested a little bit further. We are extracting a lot of information from the platform to make sure the platform is healthy.”
- Deployment Automation: “We have many clients who have different ways of executing…we needed a way to automate the solution. The most important extension for this has been our custom containers.”
Each of these extensions has added additional layers to the Goldman Sachs Enterprise Process Automation Platform, benefitting both internal teams as well as clients.
Why Developer-friendliness is Important
When selecting an automation platform, one key consideration is developer-friendliness. Often, if a solution is built with developers in mind, it’s much simpler to adopt and will be met with less resistance. What does it mean for a process automation solution to be developer friendly?
- Works within existing development environments, workflows, and best practices
- Provides APIs to allow for easy integration with other applications
- Enables teams to get started quickly with developer shortcuts, documentation and community resources.
Benefits of a Central Process Automation Platform
By now, you might be thinking about providing more structure around your organization’s automation efforts. In some cases, that might mean full centralization and governance. More often than not, your needs will likely fall somewhere along the spectrum between full centralization and full autonomy.
By adding at least some centralization to the mix, your team could experience the following benefits:
- Speed: In the models described above, a CoE would provide the resources needed for developers to quickly implement their own automation workflow. In addition, teams can save the time needed to vet a new tool for every use case.
- Scalability: Whether you’re operating at enterprise scale like Goldman Sachs or growing quickly, a cloud-based automation solution can help teams scale horizontally with ease – either by allowing teams to provision their own private cloud resources, or relying on the provider for secure hosting.
- End-to-end process orchestration: Centralizing process automation can make it much simpler to orchestrate processes from end to end. That might include orchestrating legacy systems, microservices, RPA bots, APIs, AI/ML tools, IoT devices, and more.
- Governance: From security controls to compliance regulations, centralizing on a single platform can provide the CoE with the tools they need to meet the requirements of the organization.
- IT/business stakeholder alignment: In many organizations, a CoE might serve as a bridge between technical and line-of-business stakeholders. When certain departments have an automation need, they can consult with the CoE to map out a process workflow in BPMN and/or DMN and execute it quickly.
- Optimization and Measurability: Using a central automation platform provides the CoE, developers and other stakeholders with the opportunity to understand and continuously improve business processes. Look for reporting and analytics capabilities that make it simpler to understand how end-to-end processes are functioning, and where they could be optimized.
Regardless of the path you choose to take, remember to move with intent when it comes to your process automation rollout. Following the steps detailed above, a gradual approach can prove more strategic for organizations looking to invest in long-term, end-to-end process automation and orchestration.
To get started, contact Camunda to guide you through this step-by-step process.