Scraping recipes
Step-by-step guides for scraping specific kinds of sites — with the exact settings to use.
← View all postsHow to Track an Online Store's Prices on a Daily Schedule
Target a store's product grid with a CSS selector, schedule a daily crawl, and build your own price-history tracker by diffing each day's export. A practical, honest walkthrough of what FireScraper does — and what your code does.
How to Automate Your Pipeline with the Completion Webhook
Get a signed POST the moment a crawl's exports are ready, so your own code can run automatically — embed the data, diff prices, or load a database. The event-driven glue for building a real scraping pipeline.
How to Scrape a Specific List of URLs (Not a Whole Site)
Have a hand-picked set of pages — product URLs, competitor pages, specific articles? Paste them all in and FireScraper scrapes exactly those, no crawling. One clean record per URL.
How to Monitor a Competitor's Pricing or Changelog on a Schedule
Turn any crawl into a recurring job that runs daily, weekly, or monthly — so you always have a fresh snapshot of a competitor's pricing page, a changelog, or any site you need to keep an eye on.
How to Build a Content Index of an Entire Website as JSON
Use structured extraction to pull every page's title, description, language, and word count into one clean JSON index — ideal for content audits, search indexes, and site inventories.
How to Extract Product & Listing Details into Structured JSON
Define a simple JSON schema and FireScraper pulls labeled fields — price, availability, SKU, and more — out of every page into clean, typed records. Perfect for product pages, listings, and spec sheets.
How to Extract Just the Content You Want with CSS Selectors
Use a content CSS selector to tell FireScraper exactly which part of each page to keep — the article body, a product description, or a specific component — and drop the rest of the page entirely.
How to Build a Training Corpus from a Website (Documents JSONL)
Crawl a website and export one clean JSON record per page — full text plus rich metadata — as JSONL. The ideal starting point for a fine-tuning corpus, a document store, or your own chunking pipeline.
How to Build a RAG-Ready Dataset from Any Website (Chunks JSONL)
Crawl a docs site, blog, or knowledge base and export it as pre-chunked JSONL — clean, embedding-sized passages with stable IDs and metadata, ready to drop straight into a vector database. No splitting code required.
How to Scrape a Website into a CSV Spreadsheet
Crawl an entire website and export every page as a CSV you can open in Excel, Google Sheets, or pandas — one row per page, with titles, word counts, links, and full text. No code required.
How to Save an Entire Website as Clean Markdown
Crawl any website and export every page as clean, readable Markdown — perfect for offline reading, note-taking apps like Obsidian and Notion, or feeding into an LLM.