Browse
→ Web & Browser
→ Scrapy MCP Server
Scrapy MCP Server
Official Scrapy MCP server that connects MCP-capable agents to live Scrapy crawls for inspection, debugging, and steering. It talks over stdio to the agent and over HTTP with bearer auth to Scrapy RemoteControl job files, letting agents inspect crawl progress, settings, errors, active downloads, and candidate selectors. Public GitHub repository created August 27, 2026.
MCP unverified
Integration
| Transport | stdio |
| Auth | api-key |
| Endpoint | uvx --from /absolute/path/to/scrapy-mcp-official scrapy-mcp |
| Install | git clone https://github.com/scrapy/scrapy-mcp-official && claude mcp add --scope user scrapy-mcp -- uvx --from /absolute/path/to/scrapy-mcp-official scrapy-mcp |
Use Cases
| 01 | Ask an agent whether a running Scrapy crawl is making progress or stuck and why |
| 02 | Inspect live crawl settings, middlewares, pipelines, request errors, and active downloads |
| 03 | Try candidate selectors against in-memory responses before patching a spider |
Tags
scrapy web-scraping crawler debugging remote-control python selectors stdio
Machine-readable: /api/servers.json
· JSON-LD schema embedded in <head>