Multi Agent Patterns Engineers Actually Use in Production

For production AI workflows, start with two patterns: orchestrator-worker for heterogeneous jobs with clear ownership boundaries, and plan-and-execute (a sequential pipeline with a re-planning loop) for stable, multi-step automation. Use parallel fan-out and gather when subtasks don't depend on each other. Layer in generator+critic when output quality matters more than speed, hierarchical decomposition when a single coordinator can't reasonably own the whole plan, and human-in-the-loop wherever an action is expensive to undo.

Here's the quick map we use when a team asks "which pattern first?":

• Independent, parallelizable subtasks → fan-out & gather
• A stable, known sequence of steps → plan-and-execute
• Multiple domains with distinct ownership (billing agent, infra agent, support agent) → orchestrator-worker
• Output quality is the bottleneck, not speed → generator+critic or iterative refinement
• Irreversible or high-cost actions → human-in-the-loop gate

The tradeoff underneath all of it is the same triangle every distributed systems engineer already knows: latency, cost, and complexity pull against each other, and picking a pattern without measuring against that triangle is how "smart" architectures end up slow and expensive in production.

Key Takeaways

Production multi-agent systems succeed when teams match a named pattern to real coupling and parallelism constraints, then invest in memory and task contracts before adding more patterns.

Point	Details
Start with two patterns	Orchestrator-worker for domain-owned tasks, plan-and-execute for stable multi-step automation.
Measure the payoff	Heterogeneous multi-model architectures cut latency 38.8 to 41.8% and cost 32.0 to 41.0% in benchmark testing.
Route, don't upgrade everything	Staged model routing can cut costs sharply by reserving frontier models for hard cases only.
Guard against hidden coupling	Watch for hero agents and unbounded spawning, both are cheaper to fix in design review than in production.
Memory beats pattern choice	agent-swarm applies orchestrator-worker with persistent, compounding memory across containerized workers.

Table of Contents

• What Are the Core Multi Agent Patterns?
• How Do You Match Requirements to a Pattern?
• Do Multi Agent Systems Actually Perform Better?
• What Belongs on Your Implementation Checklist?
• How Does agent-swarm.dev Apply These Patterns at Scale?
• Where Did Multi Agent Patterns Come From?
• How Do You Integrate Multi Agent Systems With What You Already Run?
• What Security Risks Are Unique to Multi-Agent Systems?
• What Tools and Frameworks Support Multi Agent Development?
• Where Are Multi Agent Patterns Headed Next?
• Why Most Teams Overinvest in Pattern Selection and Underinvest in Memory
• Get Production Multi-Agent Orchestration Without Building It From Scratch
• Sources
• FAQ

What Are the Core Multi Agent Patterns?

The Google ADK team documents eight practical patterns, and in six years of watching teams build these systems, we've never seen a production architecture that used just one. Here's the catalog, in the order we recommend evaluating them.

1. Plan-and-execute. Three roles: a planner that decomposes the goal, an executor that runs each step, and a re-planner that revises the plan when a step fails or returns unexpected data. The vanilla version re-plans after every step, which is safe but slow. Variants like ReWOO decouple planning from execution entirely (the planner writes the whole plan upfront, with placeholders for tool outputs), and LLMCompiler goes further by compiling steps into a parallelizable execution graph. Caution: vanilla plan-and-execute burns tokens re-planning on every minor deviation. Cap re-plan attempts per task.
2. Coordinator/dispatcher (orchestrator-worker). A lead agent routes incoming work to specialized workers based on domain, then merges results. Prefer this when your workers map to real organizational boundaries, like a billing agent and a deployment agent that shouldn't share context. Caution: if the coordinator starts doing the actual work instead of routing it, you've built a hero agent with extra steps.
3. Parallel fan-out & gather. The coordinator dispatches identical or related subtasks to multiple workers simultaneously, then a synthesis agent merges the outputs. This is the pattern for independent research tasks or querying multiple data sources at once. Caution: the gather step is where token costs quietly spike if you don't summarize before merging.
4. Hierarchical decomposition. Layers of coordinators, each owning a sub-goal, common in large plans where one flat coordinator would need to track too much state. Use when a single dispatcher can't reasonably hold the whole task graph in context.
5. Generator+critic. One agent produces output, a second evaluates it against explicit criteria, and the loop repeats until the critic passes it or a retry limit hits. Best for code generation, content drafting, or anything with an objective quality bar.
6. Iterative refinement. Similar to generator+critic but self-directed, a single agent revises its own output across passes. Cheaper than a two-agent loop, less rigorous.
7. Human-in-the-loop. Gate high-impact or irreversible actions behind an approval step. ADK's own guidance treats this as a first-class pattern, not an afterthought bolted onto the others.
8. Debate/voting/swarm. Multiple agents propose independent answers and a voting or debate mechanism selects the winner. Useful for ambiguous judgment calls, dangerous when the debate has no exit condition.

Pro Tip: Don't pick one pattern and force every workflow through it. ADK's own guidance notes that composite patterns, mixing pipeline, fan-out, and generator+critic in the same system, are the norm in production, not the exception.

How Do You Match Requirements to a Pattern?

Six decision axes do most of the work: coupling (how much do subtasks depend on each other?), parallelism (can steps run concurrently?), plan stability (does the sequence change based on intermediate results?), cost sensitivity (are you calling a frontier model per step?), governance and safety (what's reversible?), and ownership (does each subtask belong to a distinct domain or team?).

A compact rule set we hand to teams evaluating a new workflow:

• If subtasks are independent and order doesn't matter, use fan-out & gather.
• If the plan is known and rarely changes, use plan-and-execute with a capped re-planner.
• If subtasks map to different owners or domains, use orchestrator-worker.
• If output quality is the bottleneck, add a generator+critic loop on top of whatever base pattern you chose.
• If any step is expensive to undo, insert a human-in-the-loop gate before that step, regardless of the base pattern.

Ask stakeholders directly: "What happens if this agent is wrong and nobody catches it for an hour?" The answer tells you where the approval gates go. Watch for two red flags in early design reviews: hidden coupling (a "parallel" fan-out where workers secretly depend on shared mutable state) and the hero agent, one agent quietly absorbing responsibilities that were supposed to be distributed. Both show up as latency cliffs once you put real load through the system, and both are far cheaper to fix on a whiteboard than after launch.

Do Multi Agent Systems Actually Perform Better?

Yes, measurably. An enterprise benchmark of 200 task executions found that heterogeneous multi-model agent architectures cut end-to-end latency by a substantial percentage and operational costs by a notable margin, while pushing task success rates from 91.2 percent to 96.8 percent (p < 0.01). That's not a marginal win. It's the difference between a workflow that needs constant babysitting and one that doesn't.

Three mechanisms explain most of that gain:

• Routing. Sending routine calls to a cheaper model and only escalating hard cases to a frontier model. NVIDIA's NeMo Switchyard work reported a large cost reduction on an internal benchmark using staged routing, without giving up near-frontier accuracy.
• Fan-out. Running independent subtasks concurrently instead of serially cuts wall-clock time directly, though it multiplies your token bill if agents don't summarize before merging.
• Staged execution. Cheap, fast agents handle filtering and triage; expensive agents only see what survives that filter.

The pitfalls that erase these gains are predictable: race conditions when two workers write to shared state without coordination, token explosion from unbounded context passed between agents, and re-planning churn where a plan-and-execute loop thrashes on every minor tool-output surprise. Mitigate with idempotent task handoffs, hard token budgets per agent, and a capped re-plan counter.

What Belongs on Your Implementation Checklist?

1. Context and memory policy. Keep short-term memory scoped to the current task, not the whole session. Microsoft's guidance recommends context-limited exchanges specifically to avoid redundant token costs and context pollution between agents. Give each subagent an isolated workspace rather than a shared context window; our own breakdown of why prescriptive memory beats raw logs covers why a journal of past decisions outperforms a dump of past conversations.
2. Task contracts. Typed payloads between agents, explicit capability discovery ("agent cards" describing what a worker can and can't do), and idempotent task-state machines so a retry doesn't duplicate a side effect. Durable execution helps here too, see our notes on durable one-off script runs.
3. Security and least privilege. Microsoft's enterprise guidance recommends MCP for internal orchestration and A2A for cross-platform agent messaging, both paired with schema-validated payloads and full auditing. Never give a worker broader tool access than its specific task requires.
4. Observability and recovery. Track task states explicitly, log retries, and build reconciliation into the gather step so a partial fan-out failure doesn't silently corrupt the merged result.
5. Anti-patterns to guard against. Infinite debate loops with no exit condition, unbounded subagent spawning, and the hero agent problem we described above all appear repeatedly in field reports from the AgentPatterns catalog. We've written a longer remediation guide on agent coordination anti-patterns if you want the full list with fixes.

Pro Tip: If you can't name the exact tool permissions a worker agent has without checking three config files, you don't have least privilege. You have a hope.

How Does agent-swarm.dev Apply These Patterns at Scale?

agent-swarm runs a lead agent that decomposes objectives and assigns tasks to a standing team of specialized workers, Claude Code, Codex, OpenCode, and others, each isolated in its own container. That's orchestrator-worker as the backbone pattern, with plan-and-execute governing how the lead agent sequences multi-step objectives before handoff.

What makes it a standing swarm rather than a one-off pipeline is persistent memory: contextual knowledge compounds across runs instead of resetting every session, closer to the procedural memory architecture we've argued for elsewhere than to a stateless chat log.

• Lead agent owns decomposition and dispatch, not execution.
• Workers run isolated, so a failure in one container doesn't corrupt another's context.
• Journals persist across tasks, so the swarm doesn't relearn the same lessons every run.

The pattern only pays off if memory outlives the session. A swarm that forgets everything between tasks is just an expensive way to run one agent at a time.

Teams using agent-swarm to eliminate recurring engineering bottlenecks are documented in the case studies, including specifics on measured outcomes.

Where Did Multi Agent Patterns Come From?

Multi-agent systems didn't start in AI labs. Agent-based modeling has roots in 1990s distributed artificial intelligence and computational economics, where researchers simulated markets and ecosystems as populations of simple, interacting agents rather than one monolithic model. The core insight, that decomposing a problem into specialized, communicating agents often outperforms a single generalist agent, predates large language models by decades.

What changed with LLMs is the communication layer. Early multi-agent research relied on rigid message-passing protocols and hand-coded negotiation rules. LLM-based agents communicate in natural language or structured JSON, which makes coordination dramatically easier to build but also easier to get wrong, since a vague instruction between two agents fails silently instead of throwing a type error.

The orchestrator-worker shape itself borrows directly from distributed systems patterns that predate AI entirely: the dispatcher/worker-pool model from job queues, the map-reduce shape from batch data processing, and the supervisor-tree pattern from Erlang's OTP framework. Plan-and-execute echoes classical AI planning research from the 1970s, where a planner produced a symbolic action sequence and an executor carried it out in the world.

Recognizing these lineages matters practically. If your fan-out & gather pattern is struggling with partial failures, the fix probably already exists in map-reduce literature. You're not inventing distributed coordination from scratch. You're applying decades-old distributed systems wisdom to a new class of nondeterministic worker.

How Do You Integrate Multi Agent Systems With What You Already Run?

Most teams don't get to design a multi-agent system on a blank slate. They're bolting agent orchestration onto an existing stack: a CI/CD pipeline, a ticketing system, a Slack workspace, a data warehouse. Integration strategy matters as much as pattern choice.

The cleanest approach treats existing tools as capabilities the agent swarm calls into, rather than rebuilding those tools as agents themselves. Your GitHub repo doesn't need to become an agent. It needs an interface a worker agent can call with a well-defined, schema-validated payload. Microsoft's architecture guidance frames this as platform-native orchestration: use MCP when agents live inside one platform boundary, and A2A when you need agents on different runtimes or owned by different teams to exchange messages.

Two integration mistakes show up repeatedly. First, teams wrap every existing script in its own "agent" instead of exposing it as a tool a planner can invoke, multiplying coordination overhead for no benefit. Second, teams skip the task-state machine and let agents fire off webhooks with no idempotency guarantee, which means a retried task can double-post a Slack message or double-merge a pull request. Build the state machine before you connect the swarm to anything with real-world side effects.

For teams already running dashboards and approval workflows, the practical path is exposing those as tools the lead agent's plan can reference, not replacing them.

What Security Risks Are Unique to Multi-Agent Systems?

Governance policies (least privilege, auditing) cover the basics, but multi-agent systems introduce threat models a single-agent deployment never faces. The biggest one is prompt injection propagation: if a worker agent ingests untrusted content, a scraped webpage, a customer email, a PDF attachment, and that content contains instructions, those instructions can travel downstream to other agents that never touched the original untrusted source. A single-agent system contains the blast radius to one context window. A multi-agent system can let a poisoned instruction hop from a low-privilege research agent to a high-privilege deployment agent.

The second unique risk is inter-agent spoofing: without message authentication, a compromised or misconfigured worker can impersonate another agent's output during a gather step, corrupting the synthesis without triggering any single agent's safety checks. A2A-style protocols address this directly by requiring structured, authenticated messages between agents rather than free-form text handoffs.

Mitigation starts with treating every inter-agent message as untrusted input, not as a trusted internal signal, and validating it against a schema before the receiving agent acts on it. Segment credential scope per worker so a compromised research agent can't call a deployment tool it was never granted. Log every cross-agent handoff with enough detail to reconstruct, after the fact, exactly which agent said what to which other agent.

What Tools and Frameworks Support Multi Agent Development?

The tooling landscape splits into three layers: orchestration frameworks, communication standards, and observability platforms, and most production systems need at least one from each.

For orchestration, Google's ADK ships primitives for all eight canonical patterns directly, so you're not hand-rolling a coordinator loop from scratch. Open-source projects like the multi-model-agent reference implementation demonstrate the planner/executor split concretely, keeping workers in isolated contexts and exposing skill primitives that control what a given worker is allowed to do and how much budget it can spend.

For communication, MCP and A2A now function as the closest thing the field has to a standard: MCP for how an agent calls tools and data sources, A2A for how independent agents exchange structured messages across runtime or organizational boundaries. Teams building custom coordination testbeds should also look at experimentation platforms like Steel's Agent Games, which frames multi-agent coordination as a set of measurable game scenarios rather than a one-off internal benchmark.

Model routing deserves its own mention as a tooling category: NVIDIA's NeMo Switchyard exposes both stage-based routers and tunable routers trained on real workload data, letting you optimize the cost-quality tradeoff instead of hardcoding "always use the frontier model."

For orchestration that runs the swarm continuously rather than per-request, agent-swarm's open-source operating system handles the lead agent, container isolation, and persistent memory pieces together, rather than requiring you to wire three separate tools for orchestration, isolation, and memory.

Where Are Multi Agent Patterns Headed Next?

Model routing is getting smarter faster than pattern taxonomy is expanding. Rather than three or four canonical patterns evolving into thirty, the near-term trend is existing patterns getting better routing logic underneath them: tunable routers that learn from actual workload data will increasingly replace static, heuristic routing rules inside fan-out and plan-and-execute pipelines alike.

Composite patterns will keep winning over pure implementations. Nobody ships production plan-and-execute without some generator+critic quality gate bolted on, and that trend toward mixing patterns inside a single workflow will only deepen as teams get more comfortable with the taxonomy.

Standardization is the other clear direction. MCP and A2A are still young, but the direction of travel, structured, authenticated, schema-validated inter-agent communication replacing free-form text handoffs, mirrors exactly what happened to web services twenty years ago when SOAP and REST replaced ad hoc HTTP scraping. Expect governance tooling (auditing, credential scoping, message validation) to mature faster than pattern innovation itself over the next few years, simply because enterprise deployments won't scale past pilot stage without it.

Persistent, cross-session memory is the least mature piece of the stack today, and also the one with the most room to compound value over time for teams running the same swarm continuously rather than spinning one up per task.

Why Most Teams Overinvest in Pattern Selection and Underinvest in Memory

The conventional advice treats pattern selection as the hard problem: pick orchestrator-worker versus plan-and-execute versus fan-out, and you're most of the way to a working system. That's backwards. We've watched teams spend weeks debating the "correct" pattern for a workflow that would have worked fine under three different architectures, then ship a system that forgets everything it learned every time a container restarts.

The real bottleneck isn't taxonomy. It's memory design. A 90.2 percent-to-96.8 percent success rate improvement from heterogeneous multi-model routing is real and worth chasing, but that gain compounds only if the system retains what worked between runs. A perfectly chosen pattern running on a stateless agent is still reinventing its own wheel every session.

If you take one thing from this catalog, take this: spend your first architecture review on context and memory policy, not pattern selection. Get the task contracts and journal recall right, and honestly, most of the canonical patterns above will work well enough. Get memory wrong, and the best pattern in the world just gives you a faster way to forget.

Get Production Multi-Agent Orchestration Without Building It From Scratch

Everything above, orchestrator-worker, plan-and-execute, isolated containers, persistent memory, is available today as an open-source operating system rather than a whiteboard exercise you have to implement yourself. agent-swarm runs a lead agent that decomposes your objectives and assigns them to specialized workers (Claude Code, Codex, OpenCode, and others), each in its own isolated container, with memory that compounds across runs instead of resetting.

That last part is the piece most homegrown multi-agent builds get wrong: they solve orchestration once and rebuild memory from scratch every time a new team wants to automate a workflow. agent-swarm integrates with Slack, Linear, GitHub, and hundreds of other platforms out of the box, so the swarm plugs into tools your team already runs instead of demanding a rip-and-replace. It's self-hostable under an MIT license if you want full control, or available as a cloud-hosted subscription billed by active workers if you'd rather skip the infrastructure work entirely.

If you're comparing this against building a single-agent orchestration layer yourself, our comparison of orchestration versus accumulation walks through the architectural tradeoff directly. Otherwise, the fastest way to see the patterns in this article running against real work is to check the runnable examples and watch a live session end to end.

Sources

• Multi-Model Agentic Systems Taxonomy and Evaluation (IEEE)
• Multi-agent patterns | Microsoft Learn
• Route AI agent workloads across models with NVIDIA NeMo Switchyard

FAQ

What Is the Difference Between Orchestrator-Worker and Plan-And-Execute?

Orchestrator-worker routes tasks to specialized agents based on domain ownership, while plan-and-execute runs a single sequential plan through a planner, executor, and re-planner loop; many production systems combine both.

When Should You Use Fan-Out and Gather Instead of a Sequential Pipeline?

Use fan-out and gather when subtasks are independent and don't need each other's output, since running them concurrently cuts wall-clock latency versus a sequential pipeline that processes one step at a time.

What Causes Most Multi-Agent System Failures in Production?

Hidden coupling between supposedly independent workers, unbounded subagent spawning, and re-planning churn from an overly cautious plan-and-execute loop account for most reliability failures documented in field reports.

Is MCP or A2A Better for Multi-Agent Communication?

Neither replaces the other: MCP suits internal, platform-native tool access and orchestration, while A2A is built for cross-platform or cross-organization agent messaging with authenticated, structured payloads.

Does agent-swarm Use These Multi-Agent Patterns?

Yes, agent-swarm runs an orchestrator-worker architecture with a lead agent dispatching to isolated containerized workers, combined with plan-and-execute sequencing and persistent memory that compounds across tasks.

Recommended

• Multi-Agent Systems Reproduce Every Organizational Anti-Pattern You Already Hate | agent-swarm.dev
• Blog | agent-swarm.dev
• Your AI Workflow Has Too Many Agents | agent-swarm.dev
• The Task State Machine: 7-State Lifecycle for Recovering From Agent Crashes | agent-swarm.dev