# How to Build a Browser Agent - Tool: Browser Use - Difficulty: Intermediate - Time required: 45-60 min - Compatibility: Python 3.11+, browser-use, Playwright-managed Chromium, Claude API key, macOS/Linux/Windows - Last updated: November 2026 ## TL;DR A browser agent drives a real web browser, reading the page and clicking through it the way a person would, instead of calling a site's API. This guide builds one with the browser-use Python library and Claude that visits a list of competitor product pages, extracts price and stock as structured data, and flags anything that changed since the last run. Budget 45-60 minutes and a few cents per site checked. ## What you'll build You'll build Price Watch: an agent that visits a short list of competitor product pages, reads the current price and stock status off each one, and reports which prices moved since the last time it ran. No API, because none of these competitors publish one, which is exactly the situation a browser agent exists for. A browser agent controls a real browser through Playwright: it sees the rendered page, reads the DOM and accessibility tree, clicks elements and fills forms the way a person would, and extracts data straight out of the page rather than a fixed CSS selector that breaks the next time the site redesigns its layout. That's the whole value proposition over a traditional scraper: the agent re-plans its clicks each time based on what's actually on the page, so a reordered checkout flow or a renamed button doesn't take the whole script down. This is a different tool than a computer-use agent, which controls an entire operating system by screenshotting the full screen and clicking pixel coordinates. A browser agent only ever operates inside a browser tab and reads structured page content; it's faster, cheaper per action, and the right choice for anything with a URL. Reach for computer use only when there's no DOM at all, like a legacy desktop app. See the computer-use agent guide for that case. ## Before you start - Python 3.11+ and a terminal - An Anthropic API key (ANTHROPIC_API_KEY) with available credits - pip install browser-use, then playwright install chromium to fetch a managed browser binary - A short list of URLs you're allowed to automate against (check each site's terms before scraping it on a schedule) ## Step-by-step guide ### 1. Install browser-use and its managed browser Install the browser-use package, then run Playwright's install command to download a Chromium binary it controls directly; you don't need a system browser or a visible display for this to work. Export your Anthropic API key as an environment variable, since the LLM wrapper reads it automatically and you never want it hardcoded in a script that might end up in a scheduled job or a git repo. ``` pip install browser-use playwright install chromium export ANTHROPIC_API_KEY="sk-ant-..." ``` **Expected result:** python -c "import browser_use" runs with no errors, and a Chromium binary is present under Playwright's cache directory. ### 2. Define a structured output schema with Pydantic Instead of asking the agent to describe what it found in prose and then parsing that yourself, define the exact shape you want with a Pydantic model and pass it as the agent's output schema. The agent fills this schema from what it reads on the page, so your downstream code gets a typed object, not a paragraph you have to regex apart. ``` from pydantic import BaseModel class PriceCheck(BaseModel): title: str price: float in_stock: bool ``` **Expected result:** A Pydantic model ready to pass into browser-use's output_model_schema parameter. ### 3. Run one agent against a single product page Create an Agent with a task description that names exactly what to extract, the LLM wrapper pointed at a current Claude model, and the output schema from the previous step. Run it with the single call below and read the structured result off history.final_result() or history.structured_output. Test on one URL before you loop over the full list; browser agents are much easier to debug one page at a time. ``` from browser_use import Agent, ChatAnthropic llm = ChatAnthropic(model="claude-sonnet-5-5", temperature=0.0) async def check_one(url: str) -> PriceCheck: agent = Agent( task=f"Go to {url} and read the product title, current price, and " f"whether it's in stock. Report exactly that.", llm=llm, output_model_schema=PriceCheck, ) history = await agent.run() return history.structured_output ``` **Expected result:** A visible (or headless) Chromium window navigates to the URL, and the function returns a PriceCheck object with real title, price, and stock values. ### 4. Loop over the full list of competitor URLs Run one agent per URL in sequence rather than giving a single agent a task like 'check all five of these sites.' A fresh agent per target is more reliable: each run has a small, unambiguous task, and one site failing to load doesn't derail the rest of the batch the way a single long-running agent's context would. ``` URLS = [ "https://competitor-a.example.com/product/widget", "https://competitor-b.example.com/items/widget-pro", ] async def check_all(urls: list[str]) -> dict[str, PriceCheck]: results = {} for url in urls: results[url] = await check_one(url) return results ``` **Expected result:** A dict mapping each URL to its own PriceCheck result after running sequentially. ### 5. Compare against a stored baseline and flag changes Load the previous run's results from a JSON file, compare each current price to the stored one, and only report rows where the price actually moved. Without this step you'd have to eyeball every run's full output to notice a single competitor dropped their price 10%, which defeats the point of automating the check. ``` import json from pathlib import Path BASELINE_FILE = Path("price_baseline.json") def flag_changes(current: dict, baseline: dict) -> list[str]: alerts = [] for url, check in current.items(): old = baseline.get(url) if old and abs(check.price - old["price"]) / old["price"] > 0.02: alerts.append(f"{url}: {old['price']} -> {check.price}") return alerts ``` **Expected result:** A short list of only the URLs whose price moved more than 2% since the last saved baseline. ### 6. Harden it: retries and headless mode for unattended runs Check history.is_successful() after each run and retry once before giving up on a URL; browser agents occasionally fail a step on a slow-loading page, and a single retry recovers most of those. Run headless for any scheduled job, since there's no one watching the browser window, and log which sites failed outright so a dead competitor page doesn't silently drop out of your reports. ``` async def check_with_retry(url: str, attempts: int = 2) -> PriceCheck | None: for attempt in range(attempts): agent = Agent(task=f"Go to {url} and read price and stock.", llm=llm, output_model_schema=PriceCheck) history = await agent.run() if history.is_successful(): return history.structured_output return None ``` **Expected result:** A function that silently retries once on failure and returns None (logged, not raised) only if both attempts fail. ## Complete code example File: `price_watch.py` ```python """Price Watch: a browser agent that checks competitor prices and flags changes. Run: python price_watch.py (pip install browser-use; playwright install chromium; set ANTHROPIC_API_KEY) """ import asyncio import json from pathlib import Path from browser_use import Agent, ChatAnthropic from pydantic import BaseModel BASELINE_FILE = Path("price_baseline.json") URLS = [ "https://competitor-a.example.com/product/widget", "https://competitor-b.example.com/items/widget-pro", ] llm = ChatAnthropic(model="claude-sonnet-5-5", temperature=0.0) class PriceCheck(BaseModel): title: str price: float in_stock: bool async def check_with_retry(url: str, attempts: int = 2) -> PriceCheck | None: for _ in range(attempts): agent = Agent( task=( f"Go to {url} and read the product title, current price, and " f"whether it's in stock. Report exactly that, nothing else." ), llm=llm, output_model_schema=PriceCheck, ) history = await agent.run() if history.is_successful(): return history.structured_output return None async def check_all(urls: list[str]) -> dict[str, PriceCheck]: results: dict[str, PriceCheck] = {} for url in urls: result = await check_with_retry(url) if result is not None: results[url] = result else: print(f"[WARN] Failed to check {url} after retries") return results def load_baseline() -> dict: if BASELINE_FILE.exists(): return json.loads(BASELINE_FILE.read_text()) return {} def save_baseline(current: dict[str, PriceCheck]) -> None: serializable = {url: check.model_dump() for url, check in current.items()} BASELINE_FILE.write_text(json.dumps(serializable, indent=2)) def flag_changes(current: dict[str, PriceCheck], baseline: dict) -> list[str]: alerts = [] for url, check in current.items(): old = baseline.get(url) if old and old["price"] and abs(check.price - old["price"]) / old["price"] > 0.02: alerts.append(f"{url}: {old['price']} -> {check.price} ({check.title})") return alerts async def main(): baseline = load_baseline() current = await check_all(URLS) alerts = flag_changes(current, baseline) if alerts: print("Price changes detected:") for alert in alerts: print(f" - {alert}") else: print("No price changes above threshold.") save_baseline(current) if __name__ == "__main__": asyncio.run(main()) ``` ## Common mistakes - **Giving one agent a task like 'check all five of these competitor sites' instead of one agent per URL** — undefined Fix: Scope each agent run to a single, unambiguous task. One site timing out or redesigning its layout then only costs you that one run, not the whole batch. - **Parsing the agent's free-text summary with string matching or regex** — undefined Fix: Pass a Pydantic model as the output schema so the agent returns typed, structured data you can use directly, no parsing required. - **Running every check without ever looking at history.is_successful()** — undefined Fix: A browser agent can finish a run without actually completing the task. Check is_successful() and retry before trusting the result. - **Scraping a site on a schedule without checking its terms of service or robots.txt** — undefined Fix: Confirm you're allowed to automate against each target before you schedule repeated runs against it, and back off if a site starts blocking the traffic. - **Running headful browsers on a server with no display for a scheduled job** — undefined Fix: Run headless for any unattended job. A visible browser window only helps while you're debugging interactively. ## Best practices - Scope every agent run to one task and one target; chain multiple Agent calls instead of one agent with a long multi-site instruction. - Always define an output_model_schema for data you plan to use programmatically. Free text output is for debugging only. - Check is_successful() after every run and retry at least once before logging a failure. - Store a baseline of prior results and diff against it, so you only act on what actually changed. - Cache the Playwright browser binary in your deploy image; don't re-download it on every scheduled run. - Keep temperature at 0 for extraction tasks; you want consistent reading of the page, not creative variation. ## Use cases ### undefined ### undefined ### undefined ## Frequently asked questions ### What's the difference between a browser agent and a traditional web scraper? A traditional scraper relies on fixed CSS selectors or XPath that break the moment a site changes its layout. A browser agent reads the rendered page and accessibility tree each run and decides what to click or extract based on what's actually there, so it tolerates redesigns a selector-based script can't. ### Can a browser agent fill out forms and click buttons, not just extract data? Yes. The same Agent class that reads a price off a page can navigate multi-step flows, fill in form fields, and click through checkout or sign-up processes, since it's driving a real browser the same way a person would. ### How much does running a browser agent cost per check? Most of the cost is LLM tokens for reading page content and deciding each action, typically a few cents per page on a mid-tier model for a short, well-scoped task. A headless Chromium instance itself is free to run; you're paying for the model's reasoning over what it sees. ### Does a browser agent need a visible browser window to work? No. Playwright can run Chromium headless, with no display at all, which is what you want for any scheduled or server-side job. Run headful only while you're debugging a task interactively. ### What happens if a site blocks automated browsers? Respect it. Check the site's terms of service and robots.txt before automating against it, and back off immediately if you start seeing CAPTCHAs or blocks; that's the site telling you the traffic isn't welcome, not a technical problem to route around. ### Can someone build and host this kind of browser agent for me? Yes. RapidDev builds production browser agents, including the retry logic, scheduling, and alerting most tutorials skip. A free 30-minute consultation is the fastest way to scope what your version would need. --- Source: https://www.rapidevelopers.com/build-ai-agent/how-to-build-a-browser-agent © RapidDev — https://www.rapidevelopers.com/build-ai-agent/how-to-build-a-browser-agent