Use FireScraper in Codex: Web Data Inside Your Coding Agent
Codex is great at writing code, but it can't see the live web. When you need the docs for a library, a competitor's pricing page, or a vendor's API reference, you end up copy-pasting URLs and pages back and forth.
The FireScraper Model Context Protocol (MCP) server fixes that. Once it's connected, Codex can crawl a site, turn it into clean Markdown, and even answer questions grounded in the scraped content — all as tool calls, right inside your session.
This guide walks through installing the server, the prompts you'll actually use, and what comes back.
What you'll be able to do
- Ask Codex to crawl a URL and summarize or reason over it
- Pull a whole docs site into clean, LLM-ready Markdown
- Build a searchable corpus and ask it questions with cited sources
- Do it all without leaving your terminal
Prerequisites
- Node.js 18+ (the server runs via
npx, no install needed) - A FireScraper API key — create one on the API keys page. Keys start with
fsk_. New accounts include free crawl units, no credit card required.
Step 1 — Add the MCP server to Codex
Codex reads MCP servers from its config file at ~/.codex/config.toml. Add a firescraper entry:
[mcp_servers.firescraper]
command = "npx"
args = ["-y", "@firescraper/mcp"]
env = { FIRESCRAPER_API_KEY = "fsk_your_api_key" }
Save the file and restart Codex so it picks up the new server.
Step 2 — Confirm the tools are available
Start a Codex session and ask what FireScraper tools it can see. You should get back the FireScraper toolset. Here's what each tool does:
| Tool | What it does |
|---|---|
| firescraper_scrape_and_wait | Crawl one or more URLs and return the text in a single call — the easiest path. |
| firescraper_scrape | Start a large crawl and return a session ID immediately (poll for the result). |
| firescraper_get_session | Check a crawl's status and page counts. |
| firescraper_list_results | List the export files available for a finished crawl. |
| firescraper_get_results | Download output in a chosen format (markdown, json, chunks, vectors, …). |
| firescraper_ask_corpus | Ask a finished crawl a question and get an answer grounded in its content, with sources. |
Step 3 — Prompts you'll actually use
You don't call tools by name — you just ask. Codex picks the right tool. A few examples:
Read a single page and act on it:
Crawl https://docs.stripe.com/webhooks and summarize how signature
verification works, then write a Node example that verifies a webhook.
Codex calls firescraper_scrape_and_wait, gets clean Markdown for the page, and writes the code using the real, current docs — not its training data.
Pull a whole docs site:
Crawl https://docs.myvendor.com with depth 2 and give me the Markdown so
I can drop it into our repo as reference.
Turn a crawl into a knowledge base, then ask it:
Crawl https://help.acme.com (depth 2). Once it's done, ask the corpus:
"What is Acme's refund window and how do I request one?"
Codex runs the crawl, waits, then calls firescraper_ask_corpus and answers from the scraped pages — with a list of source URLs.
What you get back
- Markdown that preserves headings, lists, tables, and links — ready to paste into a repo or feed to the model
- Structured exports (
documents,chunks,vectors) when you're building a RAG pipeline - Grounded answers with citations from
ask_corpus, so you can trust where the answer came from
Tips
- For a quick lookup, let Codex use
firescraper_scrape_and_wait. For big sites, ask it to start the crawl and poll — it'll usefirescraper_scrape+firescraper_get_session. - Crawls use credits per page;
ask_corpususes a few credits per question. Keep an eye on your balance for large jobs. - Set a sensible depth — depth 0 is just the seed URLs; depth 2 follows links two hops in.
Give Codex eyes on the live web
Create a free FireScraper API key and connect the MCP server in two minutes.