crw

The open-source alternative to Firecrawl. One static binary, ~50 MB RAM idle, a native fastCRW REST API under /v1/* (scrape, crawl, map, search, structured extraction, and change tracking), plus a /v2/* Firecrawl compatibility layer for migrations, batch scrape, and PDF parse. Self-host free under AGPL-3.0, or hit our managed API at api.fastcrw.com. Reproducible 63.74% truth-recall on the public 1,000-URL dataset (diagnose_3way.py, 2026-05-08) — see fastcrw.com/benchmarks. Built in Rust because every millisecond of agent latency compounds.

Works with: Claude Code · Cursor · Windsurf · Cline · Copilot · Continue.dev · Codex · Gemini CLI

Why fastCRW?

• Rust-native, single static binary — no Redis, no Node.js, no Python venv, no headless-browser sidecar in the request path. One binary, one config file, one process.
• ~50 MB RAM idle — leaves headroom on a $5 VPS. Browser-render-first stacks (Firecrawl, Crawl4AI) carry a Chromium heap baseline measured in hundreds of MB before a single request lands.
• Native /v1, compatibility /v2 — new fastCRW projects should use /v1/scrape, /v1/crawl, /v1/map, and /v1/search. Existing Firecrawl v2 SDK projects can use the /v2/* compatibility layer (FirecrawlApp(api_url="https://api.fastcrw.com")) and validate the documented differences before switching production traffic.
• Change tracking & monitoring — diff a page against a prior snapshot (markdown git-diff, per-field JSON, or both) with an optional LLM "meaningful-change" judge. Stateless changeTracking primitive in the engine; scheduled monitors + signed-webhook/email alerts on the managed platform. See the Monitoring docs.
• AGPL-3.0 open core + managed option — self-host free, or point at api.fastcrw.com for managed proxy network, dashboard, and SLA without the AGPL obligations on your application code.

Comparison Table

Qualitative positioning vs. the two most-cited alternatives — the same two in the reproducible benchmark below. Numerical claims trace to the inline sources noted; everything else is descriptive.

Pricing/spec cells where claimed link to the vendor page; everything else is the qualitative architectural shape, not a comparison number.

Quickstart

Hit the managed API at api.fastcrw.com, or self-host the same binary.

# /v1/scrape — URL → markdown / HTML / JSON / links
curl -X POST https://api.fastcrw.com/v1/scrape \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","formats":["markdown"]}'

# /v1/scrape + formats:["json"] — structured JSON extraction via a JSON Schema
curl -X POST https://api.fastcrw.com/v1/scrape \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url":"https://example.com",
    "formats":["json"],
    "jsonSchema":{
      "type":"object",
      "properties":{"title":{"type":"string"}}
    }
  }'

# /v1/crawl — async multi-page job (returns a job id; poll with /v1/crawl/:id)
curl -X POST https://api.fastcrw.com/v1/crawl \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://docs.example.com","maxDepth":2,"maxPages":50}'

# Self-host (no auth, localhost) — single docker command
docker run -p 3000:3000 ghcr.io/us/crw
curl http://localhost:3000/v1/scrape \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'

Other install paths (each documented under Install further down):

npx crw-mcp                           # zero install — runs the embedded engine
pip install crw                        # Python SDK (auto-downloads binary)
brew install us/crw/crw                # Homebrew
cargo install crw-cli                  # Cargo
curl -fsSL https://fastcrw.com/install | sh

Why Rust?

Cold start is sub-second and the resident memory ceiling is bounded by the crawl queue, not by a JavaScript runtime or a headless browser parked in the background. An agent that issues N scrapes per task pays the network floor N times — anything you add on top (process spawn, JIT warmup, browser navigation overhead) multiplies. Pushing the request-path language down to Rust strips that surcharge out of every call. The same property lets one static binary saturate a $5 VPS instead of needing a multi-container compose stack, which is why the idle footprint is in the tens of MB rather than the hundreds.

MCP + SDK quickstart

fastCRW ships a built-in MCP server so any MCP-compatible agent (Claude Code, Cursor, Windsurf, Cline, Continue.dev, Codex, Gemini CLI) can call scraping tools without bespoke glue. Embedded mode runs the engine in-process — no server, no API key, no setup. The crw Python SDK and the crw-mcp Node binary both shell to the same Rust core.

npm install -g crw-mcp          # MCP server (Node wrapper)
pip install crw                 # Python SDK (auto-downloads binary)
claude mcp add crw -- npx -y crw-mcp                                       # Claude Code, embedded
claude mcp add crw \
  -e CRW_API_URL=https://api.fastcrw.com -e CRW_API_KEY=… \
  -- npx -y crw-mcp                                                        # Claude Code, managed

Per-client config recipes (Claude Desktop, Cursor, Windsurf, Cline, Continue.dev) live under docs.fastcrw.com/mcp-clients/.

Agent Skills

Beyond raw MCP tools, fastCRW ships a set of agent skills — reusable instruction packs that teach AI coding agents when and how to scrape, crawl, map, search, parse, extract, and change-track the web. Install into any agent (Claude Code, Codex, Cursor, OpenCode, Gemini