You can build a real tool-calling AI agent for $0 by running a local model through Ollama instead of paying per API call. This guide installs Ollama, pulls a tool-capable model like qwen3.8, and builds a file-organizer agent that scans a folder and sorts files by content, entirely on your own machine. It also covers free hosted tiers (Gemini's free API tier) for when local hardware isn't enough. Budget 30-45 minutes and zero dollars, plus whatever RAM your model needs.
| Fact | Value |
|---|---|
| Tool | Any LLM or framework |
| Difficulty | Beginner |
| Time required | 30-45 min |
| Compatibility | Python 3.10+, Ollama (local), 16GB+ RAM recommended, no API key or billing required, macOS/Linux/Windows |
| Last updated | November 2026 |
What you'll build
You'll build a file-organizer agent: point it at a folder and it looks at each file, decides which category it belongs in (receipts, screenshots, documents), and moves it into a matching subfolder. It runs entirely on your own machine through Ollama, no Anthropic, OpenAI, or Google API key, and no per-request bill no matter how many files you throw at it.
'Free' splits into two real paths. Local models through Ollama cost nothing per call but are bounded by your hardware and by how reliably a small model actually calls tools, qwen3.8 and Gemma 4 are the current tool-capable picks that run on a reasonable laptop. Hosted free tiers, like Gemini 3.8 Flash's free tier, cost nothing up to a quota and then either stop or start billing, and they're usually more reliable at tool calling than anything you can run locally. This guide uses the local path because it's free with no asterisk, no quota, no credit card.
Pick local when you want zero ongoing cost and your task tolerates an occasional missed or malformed tool call. Pick a hosted free tier when you need better reliability and can live within its rate limits. Graduate to a cheap paid model (Claude Haiku 5.5 or GPT-6 Luna, both well under a cent per typical call) the moment a free option's error rate costs you more time than it saves.
Prerequisites
- A machine with at least 16GB RAM (8GB can run the smallest tool-capable models, slowly)
- Ollama installed (ollama.com/download), no account or API key needed
- pip install ollama
- A test folder you don't mind an AI agent moving files around in
Step-by-step guide
Install Ollama and pull a tool-capable model
Download Ollama for your OS, then pull a model that's explicitly listed as supporting tools, not every model on Ollama does. qwen3.8 is the current well-supported pick for tool calling; if your machine is lighter on RAM, gemma4:9b trades a little tool-calling reliability for a much smaller footprint. Check ollama.com's tools filter before picking any model you haven't verified yourself.
1# Install Ollama from ollama.com/download, then:2ollama pull qwen3.83# Lighter alternative for 16GB machines:4ollama pull gemma4:9bExpected result: ollama run qwen3.8 "hello" prints a response with no network call to any paid API.
Define your tools as plain JSON schemas
Ollama's tool-calling API takes the same shape OpenAI popularized: a list of {type: 'function', function: {name, description, parameters}}. Define list_files, which returns filenames in a folder, and move_file, which moves one file into a category subfolder. There's no decorator here, you write the schema by hand, same as the from-scratch guide's raw API loop.
1TOOLS = [2 {"type": "function", "function": {3 "name": "list_files",4 "description": "List filenames in the target folder.",5 "parameters": {"type": "object", "properties": {}, "required": []},6 }},7 {"type": "function", "function": {8 "name": "move_file",9 "description": "Move a file into a category subfolder: receipts, screenshots, or documents.",10 "parameters": {11 "type": "object",12 "properties": {13 "filename": {"type": "string"},14 "category": {"type": "string", "enum": ["receipts", "screenshots", "documents"]},15 },16 "required": ["filename", "category"],17 },18 }},19]Expected result: A TOOLS list ready to pass into ollama.chat()'s tools parameter.
Write the dispatch functions
list_files reads a real directory with os.listdir(); move_file moves a file with shutil.move() into a subfolder it creates if missing. Keep both pure and testable without the model, the same discipline as the from-scratch guide.
1import os, shutil23TARGET_DIR = "./inbox"45def list_files() -> str:6 return ", ".join(os.listdir(TARGET_DIR)) or "(empty)"78def move_file(filename: str, category: str) -> str:9 dest_dir = os.path.join(TARGET_DIR, category)10 os.makedirs(dest_dir, exist_ok=True)11 shutil.move(os.path.join(TARGET_DIR, filename), os.path.join(dest_dir, filename))12 return f"Moved {filename} to {category}/"1314DISPATCH = {"list_files": list_files, "move_file": move_file}Expected result: list_files() prints real filenames from ./inbox with no model involved.
Call ollama.chat() with tools
Call ollama.chat with your model, a messages list, and the TOOLS schema. Check response['message'].get('tool_calls'); when it's present, the model wants to run one or more functions before answering. This mirrors the OpenAI-style tool-calling shape, which is what makes Ollama's API a drop-in for code you may already have written against a hosted model.
1import ollama23messages = [{"role": "user", "content": "Organize the files in my inbox folder."}]4response = ollama.chat(model="qwen3.8", messages=messages, tools=TOOLS)5messages.append(response["message"])6tool_calls = response["message"].get("tool_calls") or []Expected result: tool_calls contains one entry calling list_files before the model has any filenames to categorize.
Run the tools and loop until there's nothing left to call
For each tool call, look up the function name and arguments, run it, and append the result as a message with role 'tool'. Call ollama.chat() again with the updated messages. Repeat with a turn cap, exactly the same loop shape as the from-scratch Claude guide, just against a local model instead of a hosted API.
1for call in tool_calls:2 name = call["function"]["name"]3 args = call["function"]["arguments"]4 result = DISPATCH[name](**args)5 messages.append({"role": "tool", "content": str(result)})67response = ollama.chat(model="qwen3.8", messages=messages, tools=TOOLS)Expected result: After two rounds (list_files, then several move_file calls), the model's final message summarizes what it organized with no further tool_calls.
Know when to stop being free
Run the agent against a folder of 20-30 real files and count how often it mis-categorizes a file or skips a tool call it should have made. If the error rate is higher than you can tolerate for the task, that's the signal to move to a hosted free tier (Gemini 3.8 Flash) or a cheap paid model (Claude Haiku 5.5, GPT-6 Luna), both of which cost fractions of a cent per call and call tools far more reliably than any model you can run on a laptop today.
1# Same TOOLS and DISPATCH, swapped onto a hosted model once local reliability isn't enough:2# response = anthropic_client.messages.create(model="claude-haiku-5-5", tools=..., messages=...)Expected result: A documented decision: stay local, or move to a specific hosted model, backed by an actual error count instead of a guess.
Complete working example
1"""File-organizer agent, running entirely on a local Ollama model. Zero API cost.2Run: ollama pull qwen3.8 (once), then python file_organizer_agent.py3"""4import os5import shutil67import ollama89MODEL = "qwen3.8"10TARGET_DIR = "./inbox"11MAX_TURNS = 61213TOOLS = [14 {"type": "function", "function": {15 "name": "list_files",16 "description": "List filenames in the target folder.",17 "parameters": {"type": "object", "properties": {}, "required": []},18 }},19 {"type": "function", "function": {20 "name": "move_file",21 "description": "Move a file into a category subfolder: receipts, screenshots, or documents.",22 "parameters": {23 "type": "object",24 "properties": {25 "filename": {"type": "string"},26 "category": {"type": "string", "enum": ["receipts", "screenshots", "documents"]},27 },28 "required": ["filename", "category"],29 },30 }},31]323334def list_files() -> str:35 return ", ".join(os.listdir(TARGET_DIR)) or "(empty)"363738def move_file(filename: str, category: str) -> str:39 dest_dir = os.path.join(TARGET_DIR, category)40 os.makedirs(dest_dir, exist_ok=True)41 shutil.move(os.path.join(TARGET_DIR, filename), os.path.join(dest_dir, filename))42 return f"Moved {filename} to {category}/"434445DISPATCH = {"list_files": list_files, "move_file": move_file}4647SYSTEM_PROMPT = (48 "You organize files. Always call list_files before deciding anything. "49 "Categorize each file as receipts, screenshots, or documents based on its name, "50 "then call move_file for every file you looked at."51)525354def run(user_message: str) -> str:55 messages = [56 {"role": "system", "content": SYSTEM_PROMPT},57 {"role": "user", "content": user_message},58 ]59 for _ in range(MAX_TURNS):60 response = ollama.chat(model=MODEL, messages=messages, tools=TOOLS)61 messages.append(response["message"])62 tool_calls = response["message"].get("tool_calls") or []63 if not tool_calls:64 return response["message"]["content"]65 for call in tool_calls:66 name = call["function"]["name"]67 args = call["function"]["arguments"]68 result = DISPATCH[name](**args)69 messages.append({"role": "tool", "content": str(result)})70 raise RuntimeError("Hit MAX_TURNS without finishing")717273if __name__ == "__main__":74 os.makedirs(TARGET_DIR, exist_ok=True)75 print(run("Organize the files in my inbox folder."))76Common mistakes
Why it's a problem: Pulling any Ollama model and assuming it supports tool calling
How to avoid: Only models explicitly tagged for tools reliably return tool_calls. Check ollama.com's tools filter before building against a model you haven't verified.
Why it's a problem: Forgetting the Ollama server has to be running before ollama.chat() is called
How to avoid: Ollama runs as a local background service. If calls hang or refuse to connect, check that the Ollama app or daemon is actually running first.
Why it's a problem: Treating 'free' as 'infinitely reliable'
How to avoid: Small local models miss or malform tool calls more often than hosted flagship models. Measure your actual error rate on real inputs before trusting an unattended run.
Why it's a problem: Running file-moving tools against a real, important folder on the first test
How to avoid: Point TARGET_DIR at a throwaway test folder until you've watched the agent behave correctly across a dozen runs.
Why it's a problem: Assuming a hosted 'free tier' has no limits at all
How to avoid: Free API tiers (like Gemini's) cap requests per minute or per day and may change terms. Read the current quota before building something you depend on.
Best practices
- Pick a model from Ollama's tools-tagged list, not any model that happens to be popular.
- Match model size to your RAM: a 27B model on an 8GB machine will swap to disk and crawl.
- Keep tool handlers pure and testable without the model, same discipline as a hosted-API agent.
- Measure tool-calling error rate on real inputs before trusting a local model unattended.
- Sandbox any tool that writes or moves files to a test directory until you trust the agent's judgment.
- Keep the exact same TOOLS schema and DISPATCH functions when you eventually swap to a paid model, only the chat call changes.
Still stuck?
Copy one of these prompts to get a personalized, step-by-step explanation.
I have a local Ollama agent with list_files and move_file tools (code below) running on qwen3.8. Add a third tool, rename_file, and update the system prompt to rename files with unclear names before categorizing them. [paste complete_code]
Ask Ollama's own model in a chat: 'Given this TOOLS schema and DISPATCH dict, write a unit test for move_file that doesn't touch the real filesystem.' [paste TOOLS and DISPATCH]
Frequently asked questions
How can I create my own AI agent for free?
Run a tool-capable model locally through Ollama. There's no API key, no per-call billing, and no usage quota, the only cost is the compute and RAM already on your machine. This guide builds a complete file-organizer agent this way.
Is there a free AI agent builder with no coding required?
Several no-code platforms offer free tiers with usage caps (n8n's self-hosted Community Edition is free indefinitely; Zapier and Make cap free usage at a small number of tasks per month). See the without-code guide for a full comparison.
Which local model should I use for tool calling?
qwen3.8 is the current well-supported pick for reliable tool calling on Ollama as of late 2026. Gemma 4 (9B) is a lighter alternative for machines with less RAM, trading some reliability for a smaller footprint.
Are local models as reliable as paid APIs for tool calling?
No, not yet. Small local models miss or malform tool calls more often than flagship hosted models. That gap is the real cost of 'free', measure it on your own task before deciding it's acceptable.
What hardware do I need to run a tool-calling model locally?
16GB of RAM comfortably runs qwen3.8-class models; 8GB machines can run smaller tool-capable models like gemma4:9b, slower and with somewhat lower tool-calling accuracy.
When should I stop using a free model and pay for a hosted one?
The moment the free option's error rate costs you more time fixing mistakes than a paid model would cost in tokens. Claude Haiku 5.5 and GPT-6 Luna both run well under a cent per typical call and call tools far more reliably.
Can someone build this for me instead of doing it myself?
Yes. RapidDev builds agents on both free local models and paid APIs, and can tell you honestly which one your use case actually needs. A free 30-minute consultation is the fastest way to find out.
Talk to an Expert
Our team has built 1,000+ apps. Get personalized help with your project.
Book a free consultation