Why “one smart agent” breaks in real workflows
A single “smart agent” looks appealing when the task is one prompt and one answer. Real work is rarely that. A customer refund might require checking policy, pulling an order record, confirming payment status, drafting the response, updating a ticket, and logging the reason code. When one agent tries to hold all of that in its head, small mistakes compound: it forgets a constraint, misreads a field, or fills gaps with confident guesses.
It also struggles with accountability. If the outcome is wrong, you can’t easily tell whether the failure came from missing data, a bad tool call, or a flawed decision. The practical cost shows up as rework: humans double-check everything, or you lock the agent down so tightly it can’t finish the job.
Most “agent failures” are workflow failures in disguise: unclear handoffs, undefined decision rights, and too many steps without checkpoints.
Start with the workflow map, not the model choice
Picture the last time a process “worked” in a meeting but failed on Tuesday afternoon: an edge case, a missing field, or a person who was out sick. That’s why you start by mapping the workflow as it really runs—inputs, systems touched, decisions made, and where humans already step in—before debating which model is smartest. If you can’t draw the steps and name the artifacts (order ID, policy version, customer message, ticket status), an agent won’t magically infer them.
A useful map is specific enough that you can point to failure points. Where does the process need fresh data versus judgment? What steps are reversible (draft an email) versus risky (issue a refund)? Which tools are required, and which are “nice to have”? This also surfaces constraints that cost time and money: API rate limits, permissions reviews, and messy source data that forces humans back into the loop.
Once the map is clear, model choice becomes a sizing decision. Some steps need strong reasoning; others need reliable retrieval, structured writes, or strict validation. The workflow tells you where to spend capability, where to add checks, and where a simpler component is safer.
Common coordination patterns: manager, relay, and swarm
Teams usually coordinate work in a few recognizable ways, and multi-agent systems do best when they copy those patterns instead of inventing new ones. A “manager” pattern is a single orchestrator that breaks a request into tasks, assigns them to specialist agents (policy, payments, customer messaging), and decides when to stop and escalate. It’s slower than a one-shot prompt, but it’s easier to audit because you can see which agent made which decision and which tool calls were used.
A “relay” pattern is a pipeline: each agent does one step and hands off an artifact, like a structured refund eligibility report that the next agent must use. Relay is simple to reason about, but it’s brittle if an early step outputs sloppy or ambiguous data, so you often need validation at each handoff.
A “swarm” pattern runs several agents in parallel to propose answers, check each other, or hunt for missing data. It can catch edge cases, but it costs more (tokens, tool calls, latency) and needs a clear tie-breaker, or it turns into debate without a decision.
Defining roles, tools, and boundaries between agents
A common failure mode is giving every agent “a little access to everything” and hoping coordination emerges. Treat agents like job functions: one owns eligibility logic, another owns data lookup, another owns customer-facing language. For each role, define the inputs it can rely on, the artifact it must produce (JSON report, drafted email, updated ticket fields), and the decisions it is allowed to make. If an agent can’t name its deliverable in one sentence, it’s doing too much.
Tools should follow the same principle. Keep write-access narrow and explicit: “may create a draft,” “may update a ticket status,” “may issue a refund up to $50,” and require an approval step above that. This is where real constraints show up—permissions reviews, audit logging, API quotas, and the cost of retries when a tool times out.
Boundaries also mean stop conditions. Define when an agent must ask a human (missing order ID, policy conflict, fraud signal) and what evidence it needs to present so the handoff is fast and accountable.
Shared memory and state: what everyone must agree on
Most coordination breakdowns come from agents working off different “truths.” One agent pulls an order total that includes tax, another uses a pre-tax subtotal, and a third drafts a customer message that quietly invents a reason code. Shared memory is the small set of facts and decisions that every agent must treat as canonical: the order ID, customer identity, policy version, the current ticket status, and the working decision (“refund approved up to $X” or “needs fraud review”). Put it in a single structured state object that is written deliberately, not re-inferred from chat history.
Make ownership explicit. Decide which agent can write which fields, and require citations for any field sourced from a tool (“payment_status from Payments API at timestamp”). When state changes, log why it changed and what evidence triggered it. You’ll spend time designing schemas and handling partial updates, and you may need to redact or encrypt fields that shouldn’t be broadly visible.
Without that shared state discipline, agents will look coordinated while drifting apart, and you only notice at the final step when something irreversible happens.
Keeping multi-step runs reliable: checks, retries, and escalations
You’ve seen the pattern: the run looks fine until step seven fails, and now you’re stuck with a half-written ticket update, a duplicate refund attempt, and no clear record of what “already happened.” Reliability comes from treating each step like a production integration, not a conversation. Add checkpoints where the system must prove it has the right identifiers, the right policy version, and the right tool results before it can move forward. Keep those checks concrete: schema validation on handoff artifacts, cross-field consistency rules (amount, currency, order status), and “no missing evidence” gates for any write action.
Retries should be deliberate, not hopeful. Differentiate transient failures (timeouts, rate limits) from logical failures (missing order ID, policy conflict). Use idempotency keys for write calls so a retry can’t double-issue a refund, and cap retries to avoid loops that burn tokens and API quota. When the agent can’t resolve an issue within bounds, it should escalate with a compact packet: current state, tool logs, the decision it was trying to make, and the exact question a human needs to answer.
Escalations are also where you protect customer experience. Route them to the right queue (fraud, billing, policy) and set an SLA so “human-in-the-loop” doesn’t become “human-whenever.” If you can’t measure checkpoint failure rates, retry counts, and escalation reasons per workflow, you’ll keep debugging by reading transcripts, which doesn’t scale.
Putting it together: a practical blueprint to start small
Pick one workflow variant that’s common, low-risk, and measurable—like “refund under $50 with a valid order ID”—and build it as a relay with a single manager overseeing stop conditions. Start with read-only tools plus a “draft-only” write path, and require a structured state object at every handoff (eligibility result, evidence links, proposed action). Put one hard checkpoint before any write: validate IDs, policy version, amounts, and tool timestamps.
Ship with two metrics you’ll actually watch: percent of runs that reach a clean terminal state, and percent that escalate with a complete packet. Expect upfront costs: schema work, permissions reviews, and handling rate limits. Expand scope only after the failure modes are boring and repeatable.
