Start building
API CATEGORYDATA APIS · WEB SCRAPING

Web Scraping APIs

Turn web pages into clean, LLM-ready content: scrape a URL, crawl a site, batch-scrape pages, map a domain or extract content with Firecrawl, Exa and Tavily.

Browse 7 APIsGet API key

All Web Scraping APIs

Compare endpoints across providers, then open any API for parameters, pricing and a live request.

api
Firecrawl

Firecrawl Page Scraper

Scrapes content from target web pages with automatic handling of JavaScript rendering. Returns extracted page data based on specified formats and proxy settings.

  • web-scraping
  • content-extraction
  • markdown
PRICEFrom $0.0075
MODEapi
View endpoint
api
Firecrawl

Firecrawl Crawl

Crawl an entire website starting from a URL. Discovers and scrapes all reachable pages, returning clean markdown for each. Runs asynchronously — submit the job, then poll for results.

  • web-crawling
  • site-scraping
  • async
PRICEFrom $0.0075
MODEapi
View endpoint
api
Firecrawl

Firecrawl Batch Scrape

Scrape multiple URLs in a single batch job. Submit a list of URLs and poll for results as each page completes.

  • web-scraping
  • batch
  • async
PRICEFrom $0.0075
MODEapi
View endpoint
api
Firecrawl

Firecrawl URL Discovery

Discovers and returns URLs found across a target website. Provide an HTTPS target address and define the maximum link count to map site pages.

  • site-mapping
  • crawling
  • url-discovery
PRICEFrom $0.0075
MODEapi
View endpoint
api
Exa

Exa Contents

Extract clean page contents for a list of URLs: full text, highlights, AI summaries and metadata, from cache or with a live crawl.

  • web-scraping
  • content-extraction
PRICEFrom $0.0045
MODEapi
View endpoint
api
Tavily

Tavily URL Content Extraction

Extracts clean text content from specified web pages using their URLs. Supports basic or advanced extraction depth and optional image inclusion.

  • content-extraction
  • web-scraping
PRICEFrom $0.012
MODEapi
View endpoint
api
Tavily

Tavily Site Map

Discovers the URL structure of a target website without extracting full page content. Accepts a starting URL, maximum traversal depth, and URL discovery limits.

  • site-mapping
  • url-discovery
PRICEFrom $0.012
MODEapi
View endpoint

Endpoints on this page

The most used endpoints first.

EndpointProviderWhat it returns
Exa ContentsExaExtract clean page contents for a list of URLs: full text, highlights, AI summaries and metadata, from cache or with a live crawl
Tavily URL Content ExtractionTavilyExtracts clean text content from specified web pages using their URLs
Firecrawl Page ScraperFirecrawlScrapes content from target web pages with automatic handling of JavaScript rendering
Firecrawl URL DiscoveryFirecrawlDiscovers and returns URLs found across a target website
Tavily Site MapTavilyDiscovers the URL structure of a target website without extracting full page content
Firecrawl CrawlFirecrawlCrawl an entire website starting from a URL
Firecrawl Batch ScrapeFirecrawlScrape multiple URLs in a single batch job

Clean web content for agents and RAG

These endpoints fetch pages and return clean text or Markdown that is ready for an LLM, so you do not have to run headless browsers or HTML parsers yourself.

  • Scrape one page: Firecrawl Scrape.
  • Crawl or batch: Firecrawl Crawl and Batch Scrape.
  • Discover URLs: Firecrawl Map and Tavily Map.
  • Extract content from known URLs: Exa Contents and Tavily Extract.

To find pages first, use Web Search APIs. For structured data from specific sites, see E-commerce and Social Media APIs.

Start using Web Scraping APIs on SandBase

One key, one request pattern, billed per call.

  1. 01

    Create an API key

    Sign in to the SandBase Console and create a key on the API Keys page. Keep it in an environment variable and never commit it to source control.

    TERMINAL

    export SANDBASE_API_KEY="YOUR_SANDBASE_API_KEY"
  2. 02

    Call an endpoint

    POST a JSON body to /v1/api/firecrawl/scrape. Every endpoint on this page uses the same pattern.

    TERMINAL

    curl https://api.sandbase.ai/v1/api/firecrawl/scrape \
  -H "Authorization: Bearer $SANDBASE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "proxy": "basic"}'
  3. 03

    Let your agent call it

    Connect the SandBase MCP server or CLI so an agent can discover and call these endpoints with your key.

    TERMINAL

    npx -y @sandbaseai/cli@latest connect

Questions, answered

What format is returned?

Clean Markdown or text, plus metadata. Firecrawl can return other formats; see the endpoint page.

Can it handle JavaScript-heavy sites?

Firecrawl is built for dynamic pages, but results vary by site. Test your target pages first.

Scrape or Extract?

Use Firecrawl Scrape for full control of one page. Use Exa Contents or Tavily Extract to pull clean content from a list of URLs.

Can my agent scrape through MCP?

Yes. Connect the SandBase MCP server and the agent can call these endpoints directly.

READY / SET / RUN

Build with Web Scraping APIs.

One key, consistent APIs, and usage-based pricing across every API on this page.

Get your API keyRead the docs