A browser agent drives a real web browser, reading the page and clicking through it the way a person would, instead of calling a site's API. This guide builds one with the browser-use Python library and Claude that visits a list of competitor product pages, extracts price and stock as structured data, and flags anything that changed since the last run. Budget 45-60 minutes and a few cents per site checked.
| Fact | Value |
|---|---|
| Tool | Browser Use |
| Difficulty | Intermediate |
| Time required | 45-60 min |
| Compatibility | Python 3.11+, browser-use, Playwright-managed Chromium, Claude API key, macOS/Linux/Windows |
| Last updated | November 2026 |
What you'll build
You'll build Price Watch: an agent that visits a short list of competitor product pages, reads the current price and stock status off each one, and reports which prices moved since the last time it ran. No API, because none of these competitors publish one, which is exactly the situation a browser agent exists for.
A browser agent controls a real browser through Playwright: it sees the rendered page, reads the DOM and accessibility tree, clicks elements and fills forms the way a person would, and extracts data straight out of the page rather than a fixed CSS selector that breaks the next time the site redesigns its layout. That's the whole value proposition over a traditional scraper: the agent re-plans its clicks each time based on what's actually on the page, so a reordered checkout flow or a renamed button doesn't take the whole script down.
This is a different tool than a computer-use agent, which controls an entire operating system by screenshotting the full screen and clicking pixel coordinates. A browser agent only ever operates inside a browser tab and reads structured page content; it's faster, cheaper per action, and the right choice for anything with a URL. Reach for computer use only when there's no DOM at all, like a legacy desktop app. See the computer-use agent guide for that case.
Prerequisites
- Python 3.11+ and a terminal
- An Anthropic API key (ANTHROPIC_API_KEY) with available credits
- pip install browser-use, then playwright install chromium to fetch a managed browser binary
- A short list of URLs you're allowed to automate against (check each site's terms before scraping it on a schedule)
Step-by-step guide
Install browser-use and its managed browser
Install the browser-use package, then run Playwright's install command to download a Chromium binary it controls directly; you don't need a system browser or a visible display for this to work. Export your Anthropic API key as an environment variable, since the LLM wrapper reads it automatically and you never want it hardcoded in a script that might end up in a scheduled job or a git repo.
1pip install browser-use2playwright install chromium3export ANTHROPIC_API_KEY="sk-ant-..."Expected result: python -c "import browser_use" runs with no errors, and a Chromium binary is present under Playwright's cache directory.
Define a structured output schema with Pydantic
Instead of asking the agent to describe what it found in prose and then parsing that yourself, define the exact shape you want with a Pydantic model and pass it as the agent's output schema. The agent fills this schema from what it reads on the page, so your downstream code gets a typed object, not a paragraph you have to regex apart.
1from pydantic import BaseModel23class PriceCheck(BaseModel):4 title: str5 price: float6 in_stock: boolExpected result: A Pydantic model ready to pass into browser-use's output_model_schema parameter.
Run one agent against a single product page
Create an Agent with a task description that names exactly what to extract, the LLM wrapper pointed at a current Claude model, and the output schema from the previous step. Run it with the single call below and read the structured result off history.final_result() or history.structured_output. Test on one URL before you loop over the full list; browser agents are much easier to debug one page at a time.
1from browser_use import Agent, ChatAnthropic23llm = ChatAnthropic(model="claude-sonnet-5-5", temperature=0.0)45async def check_one(url: str) -> PriceCheck:6 agent = Agent(7 task=f"Go to {url} and read the product title, current price, and "8 f"whether it's in stock. Report exactly that.",9 llm=llm,10 output_model_schema=PriceCheck,11 )12 history = await agent.run()13 return history.structured_outputExpected result: A visible (or headless) Chromium window navigates to the URL, and the function returns a PriceCheck object with real title, price, and stock values.
Loop over the full list of competitor URLs
Run one agent per URL in sequence rather than giving a single agent a task like 'check all five of these sites.' A fresh agent per target is more reliable: each run has a small, unambiguous task, and one site failing to load doesn't derail the rest of the batch the way a single long-running agent's context would.
1URLS = [2 "https://competitor-a.example.com/product/widget",3 "https://competitor-b.example.com/items/widget-pro",4]56async def check_all(urls: list[str]) -> dict[str, PriceCheck]:7 results = {}8 for url in urls:9 results[url] = await check_one(url)10 return resultsExpected result: A dict mapping each URL to its own PriceCheck result after running sequentially.
Compare against a stored baseline and flag changes
Load the previous run's results from a JSON file, compare each current price to the stored one, and only report rows where the price actually moved. Without this step you'd have to eyeball every run's full output to notice a single competitor dropped their price 10%, which defeats the point of automating the check.
1import json2from pathlib import Path34BASELINE_FILE = Path("price_baseline.json")56def flag_changes(current: dict, baseline: dict) -> list[str]:7 alerts = []8 for url, check in current.items():9 old = baseline.get(url)10 if old and abs(check.price - old["price"]) / old["price"] > 0.02:11 alerts.append(f"{url}: {old['price']} -> {check.price}")12 return alertsExpected result: A short list of only the URLs whose price moved more than 2% since the last saved baseline.
Harden it: retries and headless mode for unattended runs
Check history.is_successful() after each run and retry once before giving up on a URL; browser agents occasionally fail a step on a slow-loading page, and a single retry recovers most of those. Run headless for any scheduled job, since there's no one watching the browser window, and log which sites failed outright so a dead competitor page doesn't silently drop out of your reports.
1async def check_with_retry(url: str, attempts: int = 2) -> PriceCheck | None:2 for attempt in range(attempts):3 agent = Agent(task=f"Go to {url} and read price and stock.", llm=llm,4 output_model_schema=PriceCheck)5 history = await agent.run()6 if history.is_successful():7 return history.structured_output8 return NoneExpected result: A function that silently retries once on failure and returns None (logged, not raised) only if both attempts fail.
Complete working example
1"""Price Watch: a browser agent that checks competitor prices and flags changes.2Run: python price_watch.py3(pip install browser-use; playwright install chromium; set ANTHROPIC_API_KEY)4"""5import asyncio6import json7from pathlib import Path89from browser_use import Agent, ChatAnthropic10from pydantic import BaseModel1112BASELINE_FILE = Path("price_baseline.json")1314URLS = [15 "https://competitor-a.example.com/product/widget",16 "https://competitor-b.example.com/items/widget-pro",17]1819llm = ChatAnthropic(model="claude-sonnet-5-5", temperature=0.0)202122class PriceCheck(BaseModel):23 title: str24 price: float25 in_stock: bool262728async def check_with_retry(url: str, attempts: int = 2) -> PriceCheck | None:29 for _ in range(attempts):30 agent = Agent(31 task=(32 f"Go to {url} and read the product title, current price, and "33 f"whether it's in stock. Report exactly that, nothing else."34 ),35 llm=llm,36 output_model_schema=PriceCheck,37 )38 history = await agent.run()39 if history.is_successful():40 return history.structured_output41 return None424344async def check_all(urls: list[str]) -> dict[str, PriceCheck]:45 results: dict[str, PriceCheck] = {}46 for url in urls:47 result = await check_with_retry(url)48 if result is not None:49 results[url] = result50 else:51 print(f"[WARN] Failed to check {url} after retries")52 return results535455def load_baseline() -> dict:56 if BASELINE_FILE.exists():57 return json.loads(BASELINE_FILE.read_text())58 return {}596061def save_baseline(current: dict[str, PriceCheck]) -> None:62 serializable = {url: check.model_dump() for url, check in current.items()}63 BASELINE_FILE.write_text(json.dumps(serializable, indent=2))646566def flag_changes(current: dict[str, PriceCheck], baseline: dict) -> list[str]:67 alerts = []68 for url, check in current.items():69 old = baseline.get(url)70 if old and old["price"] and abs(check.price - old["price"]) / old["price"] > 0.02:71 alerts.append(f"{url}: {old['price']} -> {check.price} ({check.title})")72 return alerts737475async def main():76 baseline = load_baseline()77 current = await check_all(URLS)7879 alerts = flag_changes(current, baseline)80 if alerts:81 print("Price changes detected:")82 for alert in alerts:83 print(f" - {alert}")84 else:85 print("No price changes above threshold.")8687 save_baseline(current)888990if __name__ == "__main__":91 asyncio.run(main())92Common mistakes
Why it's a problem: Giving one agent a task like 'check all five of these competitor sites' instead of one agent per URL
How to avoid: Scope each agent run to a single, unambiguous task. One site timing out or redesigning its layout then only costs you that one run, not the whole batch.
Why it's a problem: Parsing the agent's free-text summary with string matching or regex
How to avoid: Pass a Pydantic model as the output schema so the agent returns typed, structured data you can use directly, no parsing required.
Why it's a problem: Running every check without ever looking at history.is_successful()
How to avoid: A browser agent can finish a run without actually completing the task. Check is_successful() and retry before trusting the result.
Why it's a problem: Scraping a site on a schedule without checking its terms of service or robots.txt
How to avoid: Confirm you're allowed to automate against each target before you schedule repeated runs against it, and back off if a site starts blocking the traffic.
Why it's a problem: Running headful browsers on a server with no display for a scheduled job
How to avoid: Run headless for any unattended job. A visible browser window only helps while you're debugging interactively.
Best practices
- Scope every agent run to one task and one target; chain multiple Agent calls instead of one agent with a long multi-site instruction.
- Always define an output_model_schema for data you plan to use programmatically. Free text output is for debugging only.
- Check is_successful() after every run and retry at least once before logging a failure.
- Store a baseline of prior results and diff against it, so you only act on what actually changed.
- Cache the Playwright browser binary in your deploy image; don't re-download it on every scheduled run.
- Keep temperature at 0 for extraction tasks; you want consistent reading of the page, not creative variation.
Still stuck?
Copy one of these prompts to get a personalized, step-by-step explanation.
I have this browser-use price-checking agent (below) that visits competitor URLs and extracts price and stock with a Pydantic schema. Add a second check that also extracts the shipping estimate shown on the page, and alert only if the total landed cost (price plus shipping) changes by more than 2%. [paste complete_code]
In Claude Code, open price_watch.py and ask: 'Add a Slack webhook call that posts the alerts list whenever flag_changes returns anything, using an environment variable for the webhook URL.'
Frequently asked questions
What's the difference between a browser agent and a traditional web scraper?
A traditional scraper relies on fixed CSS selectors or XPath that break the moment a site changes its layout. A browser agent reads the rendered page and accessibility tree each run and decides what to click or extract based on what's actually there, so it tolerates redesigns a selector-based script can't.
Can a browser agent fill out forms and click buttons, not just extract data?
Yes. The same Agent class that reads a price off a page can navigate multi-step flows, fill in form fields, and click through checkout or sign-up processes, since it's driving a real browser the same way a person would.
How much does running a browser agent cost per check?
Most of the cost is LLM tokens for reading page content and deciding each action, typically a few cents per page on a mid-tier model for a short, well-scoped task. A headless Chromium instance itself is free to run; you're paying for the model's reasoning over what it sees.
Does a browser agent need a visible browser window to work?
No. Playwright can run Chromium headless, with no display at all, which is what you want for any scheduled or server-side job. Run headful only while you're debugging a task interactively.
What happens if a site blocks automated browsers?
Respect it. Check the site's terms of service and robots.txt before automating against it, and back off immediately if you start seeing CAPTCHAs or blocks; that's the site telling you the traffic isn't welcome, not a technical problem to route around.
Can someone build and host this kind of browser agent for me?
Yes. RapidDev builds production browser agents, including the retry logic, scheduling, and alerting most tutorials skip. A free 30-minute consultation is the fastest way to scope what your version would need.
Talk to an Expert
Our team has built 1,000+ apps. Get personalized help with your project.
Book a free consultation