Designing AI agent architecture means making five decisions before writing code: what the agent is scoped to do, whether one agent or several specialists fit, whether it's triggered synchronously or by an async event, how state survives between steps, and where a human must approve something consequential. This guide walks through all five for a shipping-exception agent, then encodes the result as a LangGraph state machine with a checkpointed pause for carrier updates. Budget 50-65 minutes.
| Fact | Value |
|---|---|
| Tool | Any LLM or framework |
| Difficulty | Advanced |
| Time required | 50-65 min |
| Compatibility | Python 3.10+, LangGraph 1.x, any model API key, any OS |
| Last updated | November 2026 |
What you'll build
You'll design and build the architecture for a shipping-exception agent: when a carrier reports a package delayed or damaged, it classifies the issue, handles delays on its own, and routes damage cases to a replacement decision that pauses for human approval before anything ships. The agent itself is almost beside the point here. The point is the five decisions you make before any node, tool, or prompt exists, because those decisions are what an architecture diagram is actually describing.
The five: (1) scope, precisely what the agent handles and what it explicitly doesn't; (2) one agent or several, does this case genuinely split into specialist roles, or is that an unnecessary layer; (3) trigger model, does work arrive as a request waiting for a reply, or as an event (a carrier webhook) that starts a case which may sit open for days; (4) state, where does the agent's memory of a case live between steps, and does it survive a restart; (5) approval, which decisions are consequential enough that the agent proposes and a human disposes. Get these wrong and no amount of prompt tuning fixes the resulting behavior.
This guide answers all five for the shipping-exception case, then encodes the answers directly as a LangGraph StateGraph: a router node that classifies the issue, two specialist nodes for delay and damage, a checkpointer so a case can sit open across multiple carrier updates, and an interrupt() call that pauses the damage path for a human's approval before a replacement ships.
Prerequisites
- Python 3.10+ and a terminal
- pip install langgraph
- A model API key (any provider; this architecture doesn't depend on one)
- Willingness to answer each of the five decisions in writing before touching code
Step-by-step guide
Decision 1: Define the scope precisely
Write down exactly what counts as a shipping exception this agent handles (delayed, lost, or damaged packages on orders already shipped) and what it explicitly doesn't (returns, pre-shipment cancellations, billing disputes). A scope stated as 'handles shipping issues' invites the agent to wander into territory you haven't designed guardrails for.
Expected result: A one-paragraph scope statement specific enough that you could point to a real case and say definitively whether it's in or out.
Decision 2: One agent or several specialists
Delay and damage cases need different judgment, delay is usually a notify-and-wait; damage usually involves money. That split justifies a router plus two specialist nodes instead of one agent holding both sets of instructions. If every case in your scope needed the same handling, a single node would be the right call. Don't split roles that don't actually diverge, see the multi-agent guide for when the opposite mistake, over-splitting, shows up.
Expected result: A short justification for exactly how many specialist paths exist and what makes each one different from the others.
Decision 3: Synchronous request or async event
A chat agent is request-response: ask, wait, get an answer. This agent is event-driven: a carrier status webhook arrives whenever it arrives, possibly days after the order shipped, and the case needs to exist and remember context across that gap. Design the trigger explicitly, a webhook handler that starts or resumes a case by order_id, rather than assuming every agent looks like a chat loop.
1# Trigger shape for this agent: an async event, not a chat turn2# POST /webhooks/carrier-status { order_id, status, reported_at }3# -> starts a new case or resumes an existing one by order_id as thread_idExpected result: A documented trigger: what external event starts or resumes work, and what identifies which case it belongs to.
Decision 4: Where state lives between steps
Because this agent can sit open for days between carrier updates, its state (order_id, issue type, carrier status history, resolution) has to survive a process restart, not just live in memory for one request. A checkpointer keyed by order_id is the mechanism; which checkpointer backend (in-memory for development, Postgres for production) is a deployment detail you can decide separately from the shape of the state itself.
1from typing import TypedDict, Literal23class CaseState(TypedDict):4 order_id: str5 issue_type: Literal["delay", "damage", "unknown"]6 carrier_status: str7 resolution: strExpected result: A typed state shape that names everything this case needs to remember, independent of which node is currently running.
Decision 5: Where a human has to approve
Notifying a customer of a delay is reversible and cheap; shipping a replacement item is not. Put the approval gate specifically on the damage path's replacement decision, not on every node, approval fatigue from gating low-stakes steps trains people to rubber-stamp everything, including the step that actually mattered.
1def handle_damage(state: CaseState) -> CaseState:2 approved = interrupt({"action": "approve_replacement", "order_id": state["order_id"]})3 if not approved:4 return {"resolution": "Replacement denied; offering store credit instead"}5 return {"resolution": f"Replacement shipped for {state['order_id']}"}Expected result: A node that stops and waits for an external decision at exactly the one point in the flow that spends money.
Encode all five decisions as a LangGraph state machine
Build the graph: a classify_issue node routes to handle_delay or handle_damage based on carrier_status, using add_conditional_edges to make the routing logic an explicit function instead of a buried if/else inside one giant node. Compile with a checkpointer so the graph can pause at handle_damage's interrupt() and resume later with the same thread_id, exactly the state-survival decision 4 called for.
1from langgraph.graph import StateGraph, START, END2from langgraph.checkpoint.memory import InMemorySaver34def route_issue(state: CaseState) -> str:5 return "handle_damage" if state["issue_type"] == "damage" else "handle_delay"67graph = StateGraph(CaseState)8graph.add_node("classify_issue", classify_issue)9graph.add_node("handle_delay", handle_delay)10graph.add_node("handle_damage", handle_damage)11graph.add_edge(START, "classify_issue")12graph.add_conditional_edges("classify_issue", route_issue, {"handle_delay": "handle_delay", "handle_damage": "handle_damage"})13graph.add_edge("handle_delay", END)14graph.add_edge("handle_damage", END)1516app = graph.compile(checkpointer=InMemorySaver())Expected result: A compiled graph where a damage case pauses mid-run, and resuming it with the same thread_id picks up exactly where it stopped.
Complete working example
1"""Shipping-exception agent architecture: router + specialists + a checkpointed2approval gate, encoding five architecture decisions as an explicit state machine.3Run: python shipping_exception_architecture.py (pip install langgraph)4"""5from typing import Literal, TypedDict67from langgraph.checkpoint.memory import InMemorySaver8from langgraph.graph import END, START, StateGraph9from langgraph.types import Command, interrupt101112class CaseState(TypedDict):13 order_id: str14 issue_type: Literal["delay", "damage", "unknown"]15 carrier_status: str16 resolution: str171819def classify_issue(state: CaseState) -> CaseState:20 # In production: call a model or rules engine on carrier_status.21 issue_type = "damage" if "damaged" in state["carrier_status"].lower() else "delay"22 return {"issue_type": issue_type}232425def handle_delay(state: CaseState) -> CaseState:26 return {"resolution": f"Notified customer of delay on {state['order_id']}"}272829def handle_damage(state: CaseState) -> CaseState:30 approved = interrupt({"action": "approve_replacement", "order_id": state["order_id"]})31 if not approved:32 return {"resolution": "Replacement denied; offering store credit instead"}33 return {"resolution": f"Replacement shipped for {state['order_id']}"}343536def route_issue(state: CaseState) -> str:37 return "handle_damage" if state["issue_type"] == "damage" else "handle_delay"383940graph = StateGraph(CaseState)41graph.add_node("classify_issue", classify_issue)42graph.add_node("handle_delay", handle_delay)43graph.add_node("handle_damage", handle_damage)44graph.add_edge(START, "classify_issue")45graph.add_conditional_edges(46 "classify_issue", route_issue, {"handle_delay": "handle_delay", "handle_damage": "handle_damage"}47)48graph.add_edge("handle_delay", END)49graph.add_edge("handle_damage", END)5051app = graph.compile(checkpointer=InMemorySaver())525354if __name__ == "__main__":55 config = {"configurable": {"thread_id": "case-8821"}}56 result = app.invoke(57 {58 "order_id": "ORD-8821",59 "carrier_status": "Package reported damaged in transit",60 "issue_type": "unknown",61 "resolution": "",62 },63 config=config,64 )65 print(result) # Paused at handle_damage's interrupt(), waiting for approval6667 # Resume the same case once a human approves the replacement:68 final = app.invoke(Command(resume=True), config=config)69 print(final["resolution"])70Common mistakes
Why it's a problem: Drawing an architecture diagram with boxes and arrows but no explicit state store behind any of it
How to avoid: Every box that needs to remember something between steps needs a named state shape and a checkpointer. If you can't say where a piece of state lives, it isn't designed yet, it's hoped for.
Why it's a problem: Defaulting to one giant agent because splitting into specialists feels like premature complexity
How to avoid: Split exactly where the judgment or tools genuinely diverge (delay vs. damage here). Don't split arbitrarily, but don't avoid splitting just to keep the diagram small.
Why it's a problem: Treating an event-driven case (a webhook that can arrive days later) like a synchronous chat turn
How to avoid: Design the trigger and the identifier that resumes a case explicitly. A chat-shaped request/response loop has no natural place to 'wait three days for a carrier update.'
Why it's a problem: Gating every node behind human approval instead of just the consequential one
How to avoid: Approval fatigue from gating cheap, reversible steps trains reviewers to rubber-stamp everything, including the replacement decision that actually needed scrutiny.
Why it's a problem: Deciding architecture and observability as separate projects
How to avoid: Bake logging of which node ran, which decision it made, and why into the graph from the first version. Retrofitting observability after an incident is much harder than designing it in.
Best practices
- Write the five decisions down in plain language before opening an editor; a diagram without written decisions behind it hides assumptions.
- Make routing logic an explicit function (add_conditional_edges), never a buried if/else inside a single oversized node.
- Choose a checkpointer backend based on whether a case needs to survive a process restart, not just whichever one is easiest to set up.
- Put approval gates on specific consequential actions, not on every step, and name exactly what triggers one in the node itself.
- Scope the agent in writing, including what it explicitly does not handle, before any node exists.
- When a production agent misbehaves, check which of the five decisions was never actually made explicit; that's usually the fastest path to the bug.
Still stuck?
Copy one of these prompts to get a personalized, step-by-step explanation.
I have a LangGraph shipping-exception agent with a router, handle_delay, handle_damage, and an interrupt()-gated approval (code below). Add a third path, handle_lost, with its own approval gate for full refunds, and update the router to classify it. [paste complete_code]
In Claude Code, open shipping_exception_architecture.py and ask: 'Add a Postgres checkpointer configuration alongside the InMemorySaver so cases survive a process restart, and add structured logging of every node transition.'
Frequently asked questions
What does AI agent architecture actually mean, beyond a diagram?
It means five concrete decisions: what the agent is scoped to do, whether it's one agent or several, whether it's triggered by a request or an async event, where its state lives between steps, and where a human has to approve. A diagram just draws the result of those decisions.
What should an AI agent architecture diagram actually show?
Nodes with their state shape, the conditions that route between them, where a checkpointer sits, and where an interrupt or approval gate exists, not just boxes labeled 'agent' and 'tools' with arrows between them.
What are the most common patterns for building AI agents?
A single tool-calling agent for well-scoped tasks, a router-plus-specialists pattern like this guide's for tasks that genuinely split, a supervisor-and-subagents pattern for broader delegation, and a retrieval-augmented pattern when the agent's job is mostly about answering from a document set.
How do I decide between a single agent and a multi-agent architecture?
Ask whether the task's phases need genuinely different tools, instructions, or review. If yes, as with this guide's delay-vs-damage split, specialists earn their complexity. If every case gets handled the same way, one well-scoped agent is simpler and cheaper.
Why does this architecture use a checkpointer instead of just storing state in a database myself?
A checkpointer is specifically built for pausing and resuming a graph mid-execution, including at an interrupt() call, with the exact in-progress state intact. You can back it with your own database, but reimplementing the pause/resume semantics yourself is the part worth avoiding.
Can someone design and build this architecture for me?
Yes. RapidDev designs and builds agent architectures on LangGraph and other frameworks, including the state, routing, and approval decisions this guide walks through. A free 30-minute consultation is the fastest way to scope your version.
Talk to an Expert
Our team has built 1,000+ apps. Get personalized help with your project.
Book a free consultation