The scraping tool built for RAG pipelines.
Turn any website into clean, structured text ready for retrieval, indexing, and evaluation pipelines. Monitor every crawl live and export in the format your stack needs.
1,000 free crawl units. No credit card required.
More than a scraper. A workspace for AI data ops.
AI-ready output
Export website content as clean text, JSONL documents, and chunked formats that slot straight into embedding, retrieval, and evaluation pipelines.
REST API, SDKs, and an MCP server
Start crawls from scripts, CI pipelines, and AI agents. TypeScript and Python SDKs, plus an MCP server that drops FireScraper straight into Claude, Cursor, and Windsurf — crawling tools right in the conversation.
Scheduled recurring crawls
Set daily, weekly, or monthly schedules from any project configuration. FireScraper queues fresh runs automatically so your datasets stay current.
Everything AI teams need from a scraper.
Parallel workers
Track every page live as it moves through the crawl pipeline with real-time queue visibility.
CSV, JSON, and JSONL exports
Download clean files ready for spreadsheets, ETL jobs, vector databases, and downstream tools.
Webhook delivery
Receive HMAC-signed callbacks when crawls finish so downstream pipelines start immediately.
Structured extraction
Define a JSON schema and pull typed fields from every page alongside the full text.
Markdown output
LLM-optimized text that uses fewer tokens than raw HTML for leaner RAG retrieval.
robots.txt support
Honour site rules automatically. Blocked URLs are logged so nothing is silently skipped.
MCP server for AI agents
Connect FireScraper to Claude, Cursor, and any MCP client with npx @firescraper/mcp — give your agent crawling tools in one line.
What teams build with FireScraper
No credit card required. Upgrade when you need more.