Why Bolting LangChain onto a Workflow Engine Backfires at Scale

At Camunda, we talk to a lot of enterprises under real pressure right now. The pressure comes from the top. Boards are asking for an AI strategy. Executives want to see agents in production, not in slide decks. Teams that have barely finished their last automation wave are being asked to ship AI-powered processes this quarter.

Most of these organizations already have workflow automation in place. Sometimes Camunda, sometimes something else, often multiple things in the same organization. So the quick-fix architecture is obvious: take the existing workflow engine and bolt AI onto it. In essence, that means calling agents at specific steps of your process model. Since you also need some abstraction over the model providers, you reach for agent frameworks like LangChain. The architecture looks like this:

The bolt-on architecture: a workflow engine calling an agent framework calling an LLM.

Shouldn't that bring you further?

Well, yes and no. Yes, because it does let you hook AI agents into existing business processes, which is at least a first step. It also looks good for the executives who need to tick the AI box.

But no, because it falls short in a handful of ways that compound as you scale. The rest of this post walks through them. The more technical problems are well-known and worth naming precisely:

1. Context lives in the agent framework and gets lost on every back and forth. When agents need to talk to humans or wait for events, you either return to the workflow engine and lose the agent's context on every round-trip, or you rebuild durable state, timers, versioning, and operational tooling inside the agent layer. We get into this in detail later, but the short version is that it's far more than just persisting a database row. Of course, you could also build your own persistence for the agent context and restore it after the workflow engine did its work, but then you're hand-rolling in bespoke code, something an orchestration platform gives you out of the box.
2. Tool governance is scattered across systems. The bolt-on architecture already has two different ways tools get invoked: in the workflow layer, calls are explicit BPMN activities you can see and govern; in the agent layer, the LLM decides dynamically which tool to call and with what arguments. Could you just let the agent use any tool it wants? No, because then you have no governance over that dynamic layer. You can tell it not to in the prompt, but a prompt is not enforcement: models have been shown to violate explicit prompted constraints. So the rule has to live somewhere outside the agent, which means tool governance ends up scattered across prompts, middleware, and custom decorators—separate from whatever governs the workflow layer, and separate from each other.
3. The audit trail spans multiple systems. Logs are scattered across at least three systems: the workflow engine, LangChain, and the LLM itself. Reconstructing what actually happened in a given case means correlating across all three.

But there are deeper fundamental issues:

4. You end up using AI in an existing process instead of rethinking the process. The biggest wins usually come from redesigning a process around the new AI capabilities from scratch, not from dropping an agent into a step of a process that was designed for a world without those capabilities. That redesign is much more practical when both deterministic and agentic steps live on the same platform so that there is no integration boundary between deterministic and agentic work.
5. The boundary between deterministic and agentic work needs to move easily, and bolt-on architectures make that expensive. As you learn more about a process, you'll want to convert agentic steps into deterministic rules, or the reverse. On a single platform, that is simply a model edit, and the technology even takes care of versioning for you. With the bolted on architecture, this grows into a complex migration project every time.

All of these are meaty topics, and we need to go deep on each. Don't expect a short post. Feel free to jump to the topics most relevant to you.

A quick note on terminology: we use LangChain throughout as shorthand for the category of tools, not the specific implementation — so we don't distinguish between LangChain and LangGraph as that distinction isn't the point of this post.

Two levels of orchestration

Before going further, I need to introduce a concept which is important to all those questions. When you introduce agents into enterprise processes, orchestration is needed at two distinct levels. We call them outer and inner orchestration.

Outer orchestration is what most people think of first. BPMN coordinates the end-to-end business journey, from initial request to final outcome, across enterprise systems, agents, and people. Agents participate in this flow, but they don't get to change its shape.

Inner orchestration is the one that tends to be missed. Here, BPMN defines the operating procedures between an agent's reasoning and its tools. Every tool invocation flows through explicit orchestration, so you can enforce guardrails, inject human review when needed, or adjust autonomy levels without modifying the agent itself. The LLM proposes what to do. The orchestration layer governs whether and how it happens.

Together, these two patterns are what we call agentic BPMN: outer orchestration coordinating the journey, inner orchestration governing the agent, both on the same engine, in the same model. It's covered in much more depth in the book I'm currently writing, Agentic Orchestration, which is available in early access (https://agentic-orchestration.io/). Figure 7 from the book, reproduced below, shows both patterns working together in a loan origination process. The orange frame is outer orchestration. The blue frame, nested inside the agent, is inner orchestration.

Inner and outer orchestration in a loan origination process. The orange frame shows outer orchestration: BPMN coordinates the end-to-end journey. The blue frame, nested inside the agent, shows inner orchestration: every tool invocation flows through the engine before it executes.

If that's still abstract, here's how it actually works in Camunda. The agent is a BPMN ad-hoc subprocess, a standard BPMN construct that contains activities which can be executed in any order, repeated, or skipped, rather than in a predefined sequence. You configure the subprocess as an AI agent by attaching a system prompt and connecting it to an LLM, whether OpenAI, Anthropic, or a local model.

The activities inside the subprocess become the agent's tools: each one is a real BPMN activity with a name and a description that the LLM uses to decide which tool to call. When the agent runs, the LLM picks the tools and the workflow engine executes them, with the durability, governance, and audit trail you'd get for any other step in your process. If you want to dig into the implementation details, the Camunda docs on AI agents are a good place to start.

To illustrate this here is a smaller loan support agent, opened in the Camunda Modeler:

A loan support agent in Camunda Modeler. The ad-hoc subprocess (purple star) is the agent's reasoning space. The boxes inside are its tools—real BPMN activities the LLM can invoke. The system prompt is visible in the properties panel, nothing is hidden.

It helps to name the parts, because "agent" means different things to different people:

• The agent is the ad-hoc subprocess (the purple star). It's the reasoning space.
• The LLM is configured in the properties panel: model provider, token limits, temperature. Here it's Claude Sonnet.
• The system prompt is plain text, visible right in the modeler. Nothing hidden behind the scenes.
• The tools are the boxes inside: preconfigured BPMN activities the agent can invoke, such as asking the customer, querying a knowledge base, consulting a loan specialist, loading available products, calculating repayments and affordability, or kicking off a full consumer loan application as its own sub-process.

Inside the subprocess, the LLM reasons over these tools and decides what to invoke and in what order. Outside it, the process is deterministic. The agent reasons, the tools act, and the process orchestrates both.

A quick side note: this pattern was introduced in Camunda 8, and at Camunda we're seeing it become the top reason Camunda 7 customers are migrating right now. Without migrating, bolt-on is your only option, and the rest of this post explains why that's not where you want to be.

One thing worth emphasizing, because it shapes a lot of what follows: when both patterns live on the same engine, you can move the boundary between deterministic and agentic behavior easily, on the same model, without rebuilding anything. We come back to this throughout the post, since it's how you actually scale agentic systems in practice.

With those two terms established, we can get into why teams reach for LangChain in the first place, and where that path starts to break down.

Why teams reach for LangChain in the first place

Let's be fair about this. LangChain became widespread for good reasons, and it's worth naming them before arguing for anything else.

The most common reason we hear is model and vendor independence. Nobody wants to hard-wire their codebase to a specific LLM provider when the landscape is shifting every quarter. An abstraction layer that lets you swap OpenAI for Anthropic for an open-source model is genuinely valuable.

Beyond that, LangChain offers useful scaffolding: prompt templating, chain composition, standardized patterns for tool calling, and ecosystem integrations. It implements common agent loop patterns so you don't have to. And it has momentum. Engineers know it, examples exist, and that matters.

We're not going to pretend none of that is useful. We are going to argue that if your goal is running agents in production at enterprise scale, LangChain sitting between your workflow engine and your LLM is the wrong place to put the orchestration responsibility. A lot of what you think you need LangChain for is already handled by a workflow engine that understands agents natively, what we call an agentic orchestration engine. The parts that genuinely require LangChain don't need to be at the coordination layer of your architecture.

What breaks bolt-on architecture

Here's the scenario that breaks the bolt-on architecture most clearly, and it shows up in almost every real use case:

Your agent needs to ask someone something. Maybe the agent is processing a claim and realizes it needs an additional document from the customer. Maybe it's drafting a response and your policy requires human approval before anything goes out. Maybe it's waiting on an email reply, a signature, a supervisor decision, or an external partner's system.

Agent frameworks have been catching up on human tasks. LangChain has a HumanInTheLoopMiddleware built on LangGraph's interrupt mechanism, with selective per-tool approval, edit, and reject decisions. There are checkpointers for state persistence. Third-party execution-governance layers like Microsoft's Agent Governance Toolkit are emerging. So yes, you can do human-in-the-loop in the agent framework now, but the result is genuinely limited.

A LangGraph checkpointer is not a durable workflow engine. It's state persistence designed for the agent's reason-act-observe loop, configured per-agent, requiring a backing store you choose and operate. A HumanInTheLoopMiddleware interrupt is not a human task. It's a pause-and-wait primitive. Real human task management means assignment to specific roles, escalations, reassignment when someone goes on leave, or work queues that load-balance across teams. You can build pieces of all this on top of LangChain interrupts. But this will be bespoke, probably per-project, and separated from whatever your business processes already use for the same purposes.

So in practice, most teams handle waiting by returning from the agent to the workflow, letting the workflow do the waiting and the human task coordination, and reinvoking the agent when the event arrives.

In the bolt-on architecture, the agent hands back to the workflow engine whenever it needs to wait. When reinvoked, it restarts from a reconstructed state—burning tokens, adding latency, and risking drift from the original reasoning.

Context gets lost with every back-and-forth

This design has a problem. When the agent gets re-invoked after the wait, it comes back fresh, with no memory of what came before. You reconstruct its state by feeding back the conversation history, the prior tool calls, and whatever else it had accumulated. The agent then resumes, or rather, restarts on the basis of that reconstruction.

This is inefficient. Every resumption means re-processing everything the agent had already reasoned through, which burns tokens and adds latency. It's expensive, because at scale those redundant tokens add up quickly. And it's potentially unsafe for correctness, because the agent doesn't actually resume where it left off and the reconstructions can drift from the original.

So teams start building context persistence themselves. You store the agent's state when it hands off to the workflow and reload it when the workflow hands back. Yes, you could lean on a checkpointer here, and yes, with enough work you can wire it through. But you're now reimplementing what an agentic orchestration engine already does, with newer primitives that haven't seen the same production hardening.

And it's not just about persisting state. Real long-running processes need timers that can fire after days or weeks, the ability to migrate in-flight instances when prompts or models change, operational dashboards that show which agents are waiting on what, escalation rules when SLAs breach, and an audit trail that holds up to a regulator. Each of these is a real engineering project on its own. The workflow engine already provides all of them, configured once for everything.

Inner orchestration puts BPMN between the LLM's decision and the tool execution. When the LLM decides to take an action, a small BPMN process fragment runs first. That fragment can wait for an external event, route to a human task queue, hold for an approval, or do anything else the situation calls for. Once the wait completes and any policy checks pass, the tool executes. Or it doesn't, depending on what came back from the wait.

With inner orchestration, the agent's full context lives in the process instance. When the human approval arrives, the engine resumes and calls the LLM again with everything still in place—no reconstruction, no restart.

You can imagine the agent's context being part of the process instance. The agent isn't a separate stateful thing that gets called by the workflow and has to be restored when the wait is over. The only thing that gets called is the LLM, and the engine passes it the context it already holds. The agent's full state, including its reasoning trace, tool call history, intermediate conclusions, and accumulated variables, lives alongside everything else the process instance knows about. When the human approval arrives three days later, the engine resumes the process and calls the LLM again with everything still in place.

Tool governance is scattered across systems

In a LangChain agent, the LLM picks which tools to call, in what order, and with what arguments. That's the whole point. The agent reasons through the situation and chooses. But "the agent decides" is also the problem, because in production you usually want some shape on those decisions: this tool only above a certain confidence score, that tool only after a human review, this combination of tools only with explicit approval, that tool never on a Sunday or in this customer segment.

The natural temptation is to just put it in the prompt. Tell the LLM "don't issue refunds above $500 without manager approval," tell it "always check fraud signals before disbursing." But prompts are suggestions and not enforced rules. The instruction holds until the LLM gets a context that confuses it, or the model gets updated, or someone slips in a prompt-injection payload. Wherever trust, compliance, or material risk is involved, you need a layer where the rule is enforced, not requested.

The stakes are not theoretical: in July 2025, a Replit AI agent deleted a live production database while under an active code freeze—despite the developer instructing it eleven times not to make changes without permission.

The agent-framework ecosystem started to build tools for this. LangChain's middleware can intercept tool calls, frameworks like Microsoft's Agent Governance Toolkit can sit between the agent and MCP tool servers, and custom decorators can wrap individual tools. These are real mechanisms, but they all share the same structural limitation: they govern only a specific agent or a specific tool. There is no place in this stack to say "wherever a refund above $500 is about to happen, stop and get approval—regardless of which agent triggered it.

In BPMN, that rule can be a reusable process fragment—a called subprocess, a shared connector, or a decision table that any process on the platform can invoke. The same approval gate that governs a refund in the claims process governs the same decision in the lending process, updated in one place and applied everywhere. It sits at the engine level, between the LLM's decision and the tool's execution, and the engine enforces it before the tool is called. And this is not a new pattern to organizations, it is just also applied for AI.

A prompt instruction is a request. A BPMN gate is enforcement. The same rule, applied at the engine level, cannot be bypassed by a context shift, a model update, or a prompt injection.

This is precisely what inner orchestration is for. When BPMN sits between the LLM's decision and the tool execution, tool governance becomes a process concern. Approval thresholds live in decision tables that a product manager can adjust without changing the agent. Compliance gates are explicit steps in the model. Rate limits are enforced by the engine. The agent still does the reasoning, while the orchestration layer governs how it acts, on the platform you already use for everything else.

The audit trail spans multiple systems

The bolt-on architecture has three separate systems that have to talk to each other, each with its own observability story.

Your workflow engine has business audit logs and execution history. LangChain has its own tracing, LangSmith. The LLM has its own usage logs. When something goes wrong, you're correlating events across three different systems to reconstruct what actually happened in a given case.

At a small scale this might be feasible, but is already annoying. At enterprise scale, where systems run millions of process instances per month, it's genuinely expensive. The engineering effort to tie these together, keep them in sync, and make the combined data queryable is non-trivial. And it compounds as every new agent deepens the observability problem.

In a bolt-on architecture, reconstructing what happened in a single case means correlating logs across three separate systems. At enterprise scale, that cost compounds with every agent added.

When your orchestration platform knows about the deterministic steps, the agentic reasoning, the tool calls, the human decisions, and the long-running waits all in one place, you get operational visibility you can't achieve any other way. Operations can see exactly where a case is, which agent step it's in, what tools have been called using which parameters, and what's waiting for human review. Compliance can see the full audit trail in business language. When a regulator asks why a specific decision was made six months ago, you pull up the process instance and the answer is there. You're not reconstructing it from correlated logs across three systems.

AI bolted-on existing processes instead of rethinking

The real payoff from AI usually comes from rethinking the whole process—what Camunda calls the Great Re-Engineering—because the constraints it was originally designed for no longer hold in a world with AI.

Consider a notice of loss process in insurance. The traditional flow exists in its current shape because structured intake was the only way to feed downstream systems reliably: a 30-field web form, document uploads, a back-office team that sorts incoming submissions, routes incomplete ones back for missing information, and queues complete ones for an adjuster. Every one of those steps was compensation for the fact that software couldn't have a conversation.

An agent changes the starting assumption entirely. A customer sends a WhatsApp message: "my car was hit in a parking lot this morning." The agent looks up the policy from the phone number, confirms coverage, asks for a photo of the damage inline, extracts the incident details from the conversation, populates the claims system, assesses severity, and routes accordingly — all before the customer has typed a tenth of what the form would have asked. None of the original steps got faster — they stopped being necessary. Each one existed because software couldn't have a conversation. Once it can, the process that replaces them looks nothing like the original.

Processes designed around human limitations look completely different when redesigned around AI capabilities.

But when the architecture makes it easy to call an agent from a specific step, the natural move is to take your existing process and drop AI into one activity. The process itself stays the same. You've added intelligence to a step that was previously rule-based or manual, and the rest of the flow continues as before. While that looks useful it is rarely where the biggest value sits and worse, it can quietly cement the existing process: now there's a working integration, a deployment, and a stakeholder who'll defend it. Redesigning the broader flow becomes harder because you have something to preserve, and because the agent is a foreign body in the process—a different runtime, a different observability story, a different set of operational properties. The more of the process you try to reshape around AI, the more time you spend maintaining the boundary between two architectures rather than improving the process itself.

When agentic behavior is native to the orchestration platform, replacing that form-based intake with a conversational agent just requires changes in a process model. You basically redraw the boundary between what's deterministic and what's agentic directly in the diagram, and the engine handles the rest.

The boundary between deterministic and agentic work needs to move

The boundary you draw when you first introduce an agent is never where it ends up. It's a bet about which parts of the work require judgment and which parts follow rules — and that bet is always provisional.

Run the insurance claims process for three months and you'll find that a large share of the WhatsApp conversations follow a pattern: straightforward fender-benders with photos, matching policy, clear liability. Those cases don't need an agent reasoning through them every time. They need a fast deterministic path with the agent reserved for the genuinely ambiguous ones. So you redraw the boundary. Or the reverse: a step you modeled as a decision table turns out to involve enough edge cases that a rule can't cover it reliably, and you hand it to an agent instead.

On a single platform, each of these is a model change. You move the boundary in the diagram, the engine handles versioning, and in-flight instances continue without disruption. The technology is designed for this kind of iterative refinement.

The boundary between deterministic and agentic work is never where you first draw it. On a single platform, moving it is a model edit. In a bolt-on architecture, it is a migration project—which is why most teams leave it where it started.

There's also a more powerful version of this pattern. Start a step as agentic when you don't yet know what shape the work takes. Run it, examine the logs, and learn which paths show up repeatedly versus which ones genuinely require judgment each time. Then convert the repeated paths into deterministic rules, leaving the agent to handle what remains. The agent becomes a discovery mechanism for what should eventually be deterministic—a way of learning the shape of a process before committing it to rules.

None of this is practical when the boundary sits between different systems. In a bolt-on setup, moving a step from the agent layer to the workflow engine or back is a migration project—two runtimes, two observability stories, an integration to maintain in between. The cost is high enough that most teams stabilize around whatever boundary they drew first.

On a single platform, that same change is a model edit. Teams that work this way tend to reshape their processes continuously as they learn. Teams on bolt-on architectures tend not to, because the cost of each change is too high to absorb routinely.

So do you still need LangChain?

It depends on what you're using it for.

If your primary reason is model and vendor independence, inner orchestration on Camunda gives you that. The agent connectors abstract the model layer. You can switch providers without rewriting your agent logic, and you can do it in the same model where your process logic lives.

If you're using LangChain for the agent loop itself—the reason-act-observe cycle with tool calling—BPMN's ad-hoc subprocess handles this natively, with the durability and governance we've been discussing.

If you're using LangChain for RAG and retrieval, Camunda has connectors for vector stores, embedding models, and the relevant pieces of the retrieval stack. You don't need LangChain for that either.

Where LangChain still has a legitimate place is in specific scaffolding tasks around AI-heavy workflows where the ecosystem has depth Camunda doesn't try to match. You can still use it for those pieces.

The point is that you don't want LangChain sitting at the coordination layer of your application, because that's where the durability, state management, and governance problems live. Use LangChain where its scaffolding helps and let the orchestration platform handle orchestration.

A fair summary

Bolting LangChain onto your existing workflow engine is the path of least resistance. It ships fast, it looks like it's working, and it does genuinely useful things. Further in, though, you tend to have an inefficiency problem when agents need to wait, a governance gap around how agents use their tools, scattered visibility across three systems, and a process that's been incrementally enhanced rather than fundamentally rethought. Often the integration itself starts to cement that incremental shape, making a broader redesign even harder the longer it runs.

Using an orchestration platform that natively supports agentic orchestration, inner and outer, asks for slightly more upfront thinking. But you avoid rebuilding things orchestration engines have already solved, and you get operational capabilities that are hard to retrofit later. And these capabilities aren't only useful for AI work—they're what any mature business process needs regardless, which means the investment carries weight beyond the AI use case alone.

Camunda doesn't replace everything LangChain does, but it does replace the parts that shouldn't have been in LangChain in the first place. The things we describe—the human-in-the-loop problem, tool governance, scattered visibility, process redesign—aren't technically impossible to handle in a bolt-on architecture. Most of them are buildable with enough effort. The argument is that the cost of building, operating, and evolving them in the agent layer compares badly to having them as native properties of the orchestration platform.

There's also an organizational cost that compounds over time. In a bolt-on architecture, process logic lives in BPMN and agent logic lives in Python—two teams, two deployment pipelines, two mental models for what the system is doing. Changes that cross the boundary require both. Incidents that cross the boundary require both teams to diagnose. On a single platform, the people who own the process can see and modify everything in one model, and the operational footprint is one runtime, not two.

A follow-up post will walk through a concrete example. We recently built both architectures for the same customer use case, one with LangChain bolted onto Camunda 7, one with Camunda 8's native inner orchestration. That post will also go into a bit more technical depth (consistency and error handling, anybody?).

If you want to read about agentic orchestration in depth, along with the four pillars of engineering trust in agentic systems, the book is in early access now: https://agentic-orchestration.io/.