# How to Build an AI Agent from Scratch - Tool: Any LLM or framework - Difficulty: Advanced - Time required: 45-60 min - Compatibility: Python 3.10+, anthropic SDK, Claude API key, any OS - Last updated: November 2026 ## TL;DR Building an AI agent from scratch means writing the tool-calling loop yourself: call the model with a tools list, check whether stop_reason is tool_use, run the matching function, send a tool_result block back, and repeat until the model answers in plain text. This guide builds a GitHub issue triage agent with two hand-defined tools against the raw Anthropic Messages API, no framework. Budget 45-60 minutes and a few cents per test run. ## What you'll build You'll build a GitHub issue triage agent: give it a repo and issue number, and it fetches the issue, decides whether it's a bug, a question, or an enhancement, and applies the label. No LangChain, no Claude Agent SDK, no OpenAI Agents SDK, just the anthropic Python package and a loop you write and own. The loop itself is short: call client.messages.create() with a tools list, check the response's stop_reason. If it's tool_use, find every tool_use content block (there can be more than one in a single response), run the matching Python function, and send the results back as tool_result blocks in a new user message. Repeat until stop_reason comes back as anything else, which means the model is done and wrote its final answer as text. Build from scratch when you need to see or control every step, when you're on a stack with no mature framework, or when you want zero dependency weight. Don't build from scratch because you distrust frameworks on principle, the loop above is exactly what create_agent, the Claude Agent SDK's query(), and the OpenAI Agents SDK's Runner all do internally, just with retries, parallel execution, and context management already handled. Once this runs, the pillar guide shows the same agent built with a framework in a third of the code. ## Before you start - Python 3.10+ and a terminal - An Anthropic API key (ANTHROPIC_API_KEY) with available credits - pip install anthropic - Comfort reading JSON Schema and a basic for-loop ## Step-by-step guide ### 1. Install the SDK and set your API key Install the anthropic package, the only dependency this guide needs. Export your API key as an environment variable; the client reads it automatically. There's no framework here to hide a missing key behind a confusing error, you'll see an authentication error immediately if it's not set. ``` pip install anthropic export ANTHROPIC_API_KEY="sk-ant-..." ``` **Expected result:** python -c "import anthropic; anthropic.Anthropic()" runs without raising. ### 2. Write the tool specs by hand as JSON Schema Without a decorator to generate it for you, you write input_schema yourself: a JSON Schema object with type, properties, and required. Define fetch_issue (repo, issue_number) and apply_label (repo, issue_number, label, constrained to an enum). This is the exact information a framework's @tool decorator would have extracted from your type hints and docstring, just written explicitly. ``` TOOLS = [ { "name": "fetch_issue", "description": "Fetch a GitHub issue's title, body, and existing labels.", "input_schema": { "type": "object", "properties": { "repo": {"type": "string"}, "issue_number": {"type": "integer"}, }, "required": ["repo", "issue_number"], }, }, { "name": "apply_label", "description": "Apply exactly one label to a GitHub issue.", "input_schema": { "type": "object", "properties": { "repo": {"type": "string"}, "issue_number": {"type": "integer"}, "label": {"type": "string", "enum": ["bug", "question", "enhancement"]}, }, "required": ["repo", "issue_number", "label"], }, }, ] ``` **Expected result:** A TOOLS list you'll pass straight into the tools parameter of every messages.create call. ### 3. Write the tool dispatch table Map each tool name to the Python function that actually performs it, separate from the schema above. Keeping fetch_issue and apply_label as plain functions means you can unit-test them without touching the model at all, something that gets harder once a framework wraps them. ``` ISSUES = {("acme/widgets", 42): {"title": "Crash on empty cart", "body": "App throws when cart has 0 items", "labels": []}} def fetch_issue(repo: str, issue_number: int) -> str: issue = ISSUES.get((repo, issue_number)) if not issue: raise ValueError(f"No issue {repo}#{issue_number}") return f"{issue['title']}: {issue['body']} (labels: {issue['labels']})" def apply_label(repo: str, issue_number: int, label: str) -> str: ISSUES[(repo, issue_number)]["labels"].append(label) return f"Applied '{label}' to {repo}#{issue_number}" DISPATCH = {"fetch_issue": fetch_issue, "apply_label": apply_label} ``` **Expected result:** DISPATCH['fetch_issue'](repo='acme/widgets', issue_number=42) returns a string with no model involved. ### 4. Write the loop: call the model and branch on stop_reason Call messages.create with your system prompt, tools, and the running messages list. stop_reason tells you what to do next: 'tool_use' means at least one tool_use block is in response.content and you must handle it before continuing; anything else (usually 'end_turn') means the model wrote its final text answer and the loop is done. Append the assistant's raw response.content to messages before you do anything else, Claude needs that exact content back in the next turn. ``` messages = [{"role": "user", "content": "Triage issue #42 in acme/widgets."}] response = client.messages.create( model="claude-sonnet-5-5", max_tokens=1024, system=SYSTEM_PROMPT, tools=TOOLS, messages=messages, ) messages.append({"role": "assistant", "content": response.content}) ``` **Expected result:** response.stop_reason prints 'tool_use' on the first call, since the model can't triage an issue it hasn't fetched yet. ### 5. Handle every tool_use block and send tool_result back A single response can contain more than one tool_use block if the model decides to call several tools in parallel. Loop over response.content, run every tool_use block through DISPATCH, and collect one tool_result per call, each tagged with that block's tool_use_id so Claude can match results to requests. Send all of them back in a single new user message, then call messages.create again. Catch exceptions here, an uncaught one crashes your process instead of telling Claude what went wrong. ``` tool_results = [] for block in response.content: if block.type != "tool_use": continue try: result = DISPATCH[block.name](**block.input) tool_results.append({"type": "tool_result", "tool_use_id": block.id, "content": str(result)}) except Exception as e: tool_results.append({ "type": "tool_result", "tool_use_id": block.id, "content": str(e), "is_error": True, }) messages.append({"role": "user", "content": tool_results}) ``` **Expected result:** A new user message containing one tool_result per tool_use_id from the previous response, ready for the next messages.create call. ### 6. Add a turn cap and run the full loop Wrap steps 4 and 5 in a for loop with a hard cap on turns (10 is plenty for this agent), the from-scratch equivalent of a framework's recursion_limit. Without it, a confused model that keeps calling tools in a cycle burns tokens indefinitely. Break out as soon as stop_reason isn't tool_use, then print the model's final text. ``` for _ in range(MAX_TURNS): response = client.messages.create(model="claude-sonnet-5-5", max_tokens=1024, system=SYSTEM_PROMPT, tools=TOOLS, messages=messages) messages.append({"role": "assistant", "content": response.content}) if response.stop_reason != "tool_use": break # ... build and append tool_results as in step 5 ... else: raise RuntimeError("Hit MAX_TURNS without finishing") ``` **Expected result:** The loop exits on its own once the model fetches the issue, applies a label, and reports back, well under the turn cap. ## Complete code example File: `triage_agent.py` ```python """GitHub issue triage agent, built from scratch against the raw Messages API. Run: python triage_agent.py (pip install anthropic; set ANTHROPIC_API_KEY) """ import anthropic client = anthropic.Anthropic() MAX_TURNS = 10 ISSUES = { ("acme/widgets", 42): { "title": "Crash on empty cart", "body": "App throws an unhandled exception when the cart has 0 items.", "labels": [], } } TOOLS = [ { "name": "fetch_issue", "description": "Fetch a GitHub issue's title, body, and existing labels.", "input_schema": { "type": "object", "properties": {"repo": {"type": "string"}, "issue_number": {"type": "integer"}}, "required": ["repo", "issue_number"], }, }, { "name": "apply_label", "description": "Apply exactly one label to a GitHub issue.", "input_schema": { "type": "object", "properties": { "repo": {"type": "string"}, "issue_number": {"type": "integer"}, "label": {"type": "string", "enum": ["bug", "question", "enhancement"]}, }, "required": ["repo", "issue_number", "label"], }, }, ] def fetch_issue(repo: str, issue_number: int) -> str: issue = ISSUES.get((repo, issue_number)) if not issue: raise ValueError(f"No issue {repo}#{issue_number}") return f"{issue['title']}: {issue['body']} (labels: {issue['labels']})" def apply_label(repo: str, issue_number: int, label: str) -> str: ISSUES[(repo, issue_number)]["labels"].append(label) return f"Applied '{label}' to {repo}#{issue_number}" DISPATCH = {"fetch_issue": fetch_issue, "apply_label": apply_label} SYSTEM_PROMPT = ( "You triage GitHub issues. Always fetch the issue before labeling it. " "Apply exactly one label: bug, question, or enhancement." ) def run(user_message: str) -> str: messages = [{"role": "user", "content": user_message}] for _ in range(MAX_TURNS): response = client.messages.create( model="claude-sonnet-5-5", max_tokens=1024, system=SYSTEM_PROMPT, tools=TOOLS, messages=messages, ) messages.append({"role": "assistant", "content": response.content}) if response.stop_reason != "tool_use": return "".join(b.text for b in response.content if b.type == "text") tool_results = [] for block in response.content: if block.type != "tool_use": continue try: result = DISPATCH[block.name](**block.input) tool_results.append({"type": "tool_result", "tool_use_id": block.id, "content": str(result)}) except Exception as e: tool_results.append({ "type": "tool_result", "tool_use_id": block.id, "content": str(e), "is_error": True, }) messages.append({"role": "user", "content": tool_results}) raise RuntimeError("Hit MAX_TURNS without finishing") if __name__ == "__main__": print(run("Triage issue #42 in acme/widgets.")) ``` ## Common mistakes - **Appending only the final text instead of the full response.content list to messages** — undefined Fix: The next turn needs the exact tool_use blocks back in history. Always append response.content as returned, not a reconstructed version. - **Assuming a response has at most one tool_use block** — undefined Fix: Claude can request several tool calls in one response. Loop over every block in response.content and send back one tool_result per tool_use_id. - **Running the loop with no turn cap** — undefined Fix: A model stuck re-calling a tool with the same bad input will loop until you hit a context limit or a bill you didn't expect. Cap turns and raise on exceeding it. - **Writing input_schema without "type": "object" at the top level** — undefined Fix: Every tool's input_schema must be a JSON Schema object type with a properties map. A malformed schema fails at request time, not quietly. - **Letting a tool handler's exception propagate and crash the process** — undefined Fix: Catch it, set is_error: true on the tool_result, and let Claude see what went wrong. It can often recover (retry with different input, ask the user) if you give it the chance. ## Best practices - Keep tool handlers as plain, independently testable functions; the loop should only call DISPATCH[name](**input), never contain business logic. - Always set is_error: true on a failed tool_result instead of stuffing an exception message into a normal one. - Cap turns explicitly and treat hitting the cap as a bug report, not an edge case to silently swallow. - Log every request and response pair to disk during development; replaying a failed run without re-calling the API saves real debugging time. - State tool sequencing rules in the system prompt as explicitly as you would in code comments ('always fetch before labeling'). - Move to a framework the moment you're reimplementing retries, parallel tool execution, or context trimming that the framework already ships. ## Use cases ### undefined ### undefined ### undefined ## Frequently asked questions ### Why build an AI agent from scratch instead of using a framework? To see and control every step of the loop, to run on a stack with no mature framework yet, or to keep the dependency footprint at just the model's own SDK. Most production teams still move to a framework once the agent grows past one or two tools. ### What exactly is a 'tool-calling loop'? The cycle of calling the model with a tools list, checking whether it wants to call one (stop_reason == 'tool_use'), running that tool, sending the result back as a tool_result block, and repeating until the model answers in plain text instead. ### What happens if I forget to send a tool_result for a tool_use block? The next messages.create call fails or the model gets confused, because it's still waiting to see the result of a tool call it made. Every tool_use_id in a response must get exactly one matching tool_result in the next message. ### Can a from-scratch agent call multiple tools at once? Yes, Claude can return several tool_use blocks in a single response when it decides the calls are independent. Your loop needs to handle all of them, not just the first, or later calls silently get skipped. ### How is this different from what LangChain's create_agent does internally? Functionally, almost nothing, create_agent runs the same request-response-tool_result cycle. It adds retries, parallel tool execution, context trimming, and a checkpointer you'd otherwise write yourself, which is the entire case for using it once the agent gets more complex. ### Can someone build this for me instead of doing it myself? Yes. RapidDev builds production agents both from scratch and on frameworks like LangChain and the Claude Agent SDK, including the retry and audit-trail logic this guide leaves as an exercise. A free 30-minute consultation is the fastest way to scope your version. --- Source: https://www.rapidevelopers.com/build-ai-agent/how-to-build-an-ai-agent-from-scratch © RapidDev — https://www.rapidevelopers.com/build-ai-agent/how-to-build-an-ai-agent-from-scratch