Skip to main content
RapidDev - Software Development Agency
mcp-tutorial

How to use the Puppeteer MCP server for web scraping

The Puppeteer/Playwright MCP server gives your AI assistant the ability to control a real web browser. It can navigate to URLs, take screenshots, click buttons, fill forms, extract page content, and run JavaScript on any website. This is powerful for web scraping, testing, monitoring, and debugging — your AI can see and interact with web pages just like a human user, all through natural language commands in your chat window.

What you'll learn

  • How to install and configure the Playwright MCP server for browser automation
  • How to navigate pages, take screenshots, and extract content through AI
  • How to fill forms, click buttons, and interact with web applications
  • How to use browser automation for testing and debugging workflows
Book a free consultation
4.9Clutch rating
600+Happy partners
17+Countries served
190+Team members
Intermediate7 min read15 minutesClaude Desktop, Cursor, Windsurf, VS Code (Copilot)March 2026RapidDev Engineering Team
TL;DR

The Puppeteer/Playwright MCP server gives your AI assistant the ability to control a real web browser. It can navigate to URLs, take screenshots, click buttons, fill forms, extract page content, and run JavaScript on any website. This is powerful for web scraping, testing, monitoring, and debugging — your AI can see and interact with web pages just like a human user, all through natural language commands in your chat window.

Control a Real Web Browser from Your AI Assistant

The @anthropic-ai/playwright-mcp server connects your AI assistant to a real browser instance powered by Playwright. Unlike web search which returns text snippets, this server lets the AI actually visit web pages, see their rendered content, take screenshots, interact with elements, fill forms, and execute JavaScript. This opens up powerful workflows: the AI can test your web application, scrape data from sites, monitor page changes, fill out forms on your behalf, and debug rendering issues — all by controlling a browser through natural language instructions.

Prerequisites

  • Node.js 18 or later installed on your machine
  • Claude Desktop, Cursor, or another MCP-compatible AI host
  • A display environment (the browser window will be visible on your screen)
  • Basic understanding of web pages and HTML elements

Step-by-step guide

1

Add the Playwright MCP server to your configuration

Open your MCP host's configuration file and add a playwright entry. The server uses the @anthropic-ai/playwright-mcp package, which is the official Anthropic-maintained browser automation server. It uses npx with the -y flag for automatic installation. On first run, Playwright will download the browser binaries it needs (Chromium by default).

typescript
1{
2 "mcpServers": {
3 "playwright": {
4 "command": "npx",
5 "args": [
6 "-y",
7 "@anthropic-ai/playwright-mcp"
8 ]
9 }
10 }
11}

Expected result: Your configuration file contains the Playwright MCP server entry.

2

Restart your AI host and verify the connection

Fully quit and reopen Claude Desktop, Cursor, or VS Code. The Playwright server will start and its browser automation tools will become available. You should see tools for navigation, screenshots, clicking, typing, and JavaScript execution in the tools list.

Expected result: The Playwright MCP server shows as connected with browser automation tools available.

3

Navigate to a web page and take a screenshot

Start with a simple navigation task. Ask the AI to visit a URL and describe what it sees, or take a screenshot. The AI will launch a browser window (visible on your screen), navigate to the page, and capture the rendered result. This confirms the server is working correctly.

typescript
1Example prompts:
2
3"Navigate to https://example.com and take a screenshot"
4
5"Go to my app at localhost:3000 and describe the current state of the homepage"
6
7"Visit https://news.ycombinator.com and tell me the top 5 stories right now"

Expected result: A browser window opens, navigates to the URL, and the AI describes the page content or returns a screenshot.

4

Interact with page elements — click buttons and fill forms

The AI can interact with web page elements by clicking buttons, filling input fields, selecting dropdowns, and submitting forms. Describe the action you want in natural language and the AI will find the right element on the page and interact with it.

typescript
1Example prompts:
2
3"Go to localhost:3000/login, fill in the email field with 'test@example.com' and the password with 'password123', then click the Sign In button"
4
5"Navigate to my app, click the 'Create New Project' button, and fill out the form with a sample project name"
6
7"Go to localhost:3000/settings and toggle the dark mode switch"

Expected result: The AI clicks, types, and interacts with page elements, reporting the results of each action.

5

Extract content and run JavaScript on pages

The AI can extract text content from specific page elements, scrape structured data, and execute custom JavaScript in the browser context. This is useful for testing, debugging, and data extraction. The AI can inspect the DOM, check console errors, and verify page behavior.

typescript
1Example prompts:
2
3"Go to localhost:3000 and check if there are any JavaScript errors in the browser console"
4
5"Navigate to localhost:3000/dashboard and extract all the data from the stats cards into a table"
6
7"Visit my app, open the browser dev tools network tab, click the Submit button, and tell me what API calls are made"

Expected result: The AI extracts content, runs JavaScript, and reports findings from the browser session.

6

Configure for VS Code with the servers key

For VS Code with GitHub Copilot, use the servers key instead of mcpServers. Create a .vscode/mcp.json file in your workspace root.

typescript
1{
2 "servers": {
3 "playwright": {
4 "command": "npx",
5 "args": [
6 "-y",
7 "@anthropic-ai/playwright-mcp"
8 ]
9 }
10 }
11}

Expected result: VS Code recognizes the Playwright MCP server and Copilot can control a browser.

Complete working example

claude_desktop_config.json
1{
2 "mcpServers": {
3 "playwright": {
4 "command": "npx",
5 "args": [
6 "-y",
7 "@anthropic-ai/playwright-mcp"
8 ]
9 }
10 }
11}
12
13// VS Code variant (.vscode/mcp.json):
14// {
15// "servers": {
16// "playwright": {
17// "command": "npx",
18// "args": ["-y", "@anthropic-ai/playwright-mcp"]
19// }
20// }
21// }
22
23// No API keys or environment variables required.
24// Playwright downloads Chromium automatically on first run.
25
26// Available tools:
27// - navigate: Go to a URL
28// - screenshot: Capture the current page
29// - click: Click an element by selector or text
30// - fill: Type text into an input field
31// - select: Choose a dropdown option
32// - evaluate: Run JavaScript in the browser
33// - get_text: Extract text content from elements
34// - hover: Hover over an element
35// - scroll: Scroll the page
36// - wait: Wait for an element or condition
37// - go_back / go_forward: Browser navigation

Common mistakes when using the Puppeteer MCP server for web scraping

Why it's a problem: Running the server in a headless environment without a display

How to avoid: The default mode launches a visible browser window. If you need headless mode (no visible window), check the server documentation for a --headless flag or environment variable. Some CI environments may need Xvfb for display emulation.

Why it's a problem: Expecting the AI to handle complex multi-page authentication flows

How to avoid: OAuth flows, CAPTCHAs, and multi-factor authentication are difficult for AI to handle. Log in manually first and let the AI work with the authenticated session, or use browser cookies/session storage to skip login.

Why it's a problem: Not waiting for page loads before interacting with elements

How to avoid: When asking the AI to click or fill elements right after navigation, the page may not be fully loaded. Ask the AI to wait for the element to appear before interacting: 'Navigate to the page and wait for the login form to appear, then fill it out.'

Why it's a problem: Using the Playwright MCP server for high-volume scraping

How to avoid: The MCP server is designed for interactive use, not automated bulk scraping. Each page visit counts as AI context. For high-volume scraping, use a dedicated scraping tool and bring the results into the AI via the filesystem MCP server.

Best practices

  • Start with navigation and screenshots to verify the server works before complex interactions
  • Use the server to test your own applications at localhost rather than external sites
  • Ask the AI to describe what it sees on the page before performing actions
  • Combine with the filesystem MCP server to save screenshots and extracted data to files
  • Use specific selectors or element descriptions when telling the AI what to click
  • Close the browser session when done to free resources
  • Test form submissions on development environments, not production
  • Use the JavaScript evaluation tool for debugging — check console errors and DOM state

Still stuck?

Copy one of these prompts to get a personalized, step-by-step explanation.

ChatGPT Prompt

I want to set up the Playwright MCP server so my AI assistant can control a web browser. Walk me through configuring it in Claude Desktop and show me how to navigate to a page, take a screenshot, and interact with form elements.

MCP Prompt

Navigate to localhost:3000, take a screenshot of the homepage, then go to the /signup page, fill out the registration form with test data, submit it, and tell me if the signup was successful or if there were any errors.

Frequently asked questions

Does the Playwright MCP server work in headless mode?

By default, it launches a visible browser window so the AI can take meaningful screenshots. Some implementations support a headless flag for running without a visible window, which is useful for server environments.

Can the AI handle CAPTCHAs and bot detection?

No. CAPTCHAs are specifically designed to block automated browsers. Sites with aggressive bot detection may also block the Playwright browser. The server works best with your own applications and sites that do not employ bot prevention.

Does it use Puppeteer or Playwright?

The @anthropic-ai/playwright-mcp server uses Playwright, which is Microsoft's browser automation library. It supports Chromium, Firefox, and WebKit. An older Puppeteer-based MCP server also exists in the community, but the Anthropic-maintained Playwright version is recommended.

Can I use this to test my web application?

Yes. This is one of the best use cases. Point the AI at your localhost development server and ask it to test forms, navigation, error states, and responsive layouts. It can report issues, take screenshots of bugs, and even suggest fixes.

Is web scraping with this server legal?

Web scraping legality depends on the site's terms of service and your jurisdiction. Scraping your own applications is always fine. For third-party sites, check their robots.txt and terms of service. The MCP server is a tool — you are responsible for how you use it.

Can RapidDev help set up automated browser testing with MCP?

Yes. RapidDev can help you build an AI-powered testing workflow that combines browser automation with your CI/CD pipeline, using MCP servers for interactive debugging and Playwright directly for automated regression tests.

RapidDev

Talk to an Expert

Our team has built 600+ apps. Get personalized help with your project.

Book a free consultation

Need help with your project?

Our experts have built 600+ apps and can accelerate your development. Book a free consultation — no strings attached.

Book a free consultation

We put the rapid in RapidDev

Need a dedicated strategic tech and growth partner? Discover what RapidDev can do for your business! Book a call with our team to schedule a free, no-obligation consultation. We'll discuss your project and provide a custom quote at no cost.