Fitter — web data for AI agents

Fitter turns any website or API into structured JSON — declaratively. One JSON/YAML config describes where the data lives (HTTP request, headless browser, file, static value) and what to extract (JSON paths, CSS selectors, XPath). No code, no brittle scraping scripts.

Because configs are plain data, LLMs can author them. The built-in MCP server lets Claude Code, Claude Desktop, or any MCP client write and run scraping pipelines on your machine, on demand:

"Get the top 5 HackerNews stories with titles and scores" → the model authors a fitter config, validates it, runs it locally, and gets clean JSON back.

One engine, five ways to use it:

Why fitter for AI agents?

• Declarative & auditable — the agent produces a config you can read, save and re-run, not throwaway code
• Local-first — all fetching happens on your machine; no third-party scraping API, no keys, no per-request billing
• Batteries included — HTTP client, headless browser (Playwright/Chromium/Docker), JSON/HTML/XML/XPath parsing, pagination, cached references, host rate-limits — in a single static binary
• Reusable — what the agent authored today becomes tomorrow's cron job or service config

How to use Fitter_MCP

Fitter MCP is a Model Context Protocol server (stdio transport) which lets any MCP client — Claude Code, Claude Desktop, IDE assistants, custom agents — run Fitter configs and get structured JSON back.

Quick start (Claude Desktop — one click)

Download fitter-mcp-<os>-<arch>.mcpb from the release page and open it — Claude Desktop installs the server automatically.

Quick start (Claude Code)

# 1. get the binary: download fitter_mcp_<version>-<os>-<arch> from the release page
#    https://github.com/PxyUp/fitter/releases — or build it from source:
go build -o fitter_mcp ./cmd/mcp

# 2. register it once, available in every project
claude mcp add fitter -s user -- "$(pwd)/fitter_mcp"

Then just ask:

Get the top 5 HackerNews stories with titles and scores using fitter

The model calls fitter_config_reference, authors a config, optionally checks it with fitter_validate_config and executes it via fitter_run — all data fetching happens locally on your machine. For a ready-made pipeline try examples/config_morning_briefing.json:

Run examples/config_morning_briefing.json with fitter and give me the briefing

Register in Claude Desktop

{
  "mcpServers": {
    "fitter": {
      "command": "/path/to/fitter_mcp"
    }
  }
}

Tools

The reference is also exposed as MCP resource fitter://config-reference for clients that support resources.

The config format is exactly the same as for Fitter_CLI: a top-level object with item (required), limits and references. Notifiers work too (the result is additionally pushed to http/telegram/redis/file/console); trigger_config and http_server are service-mode only and are ignored in MCP calls.

Remote / hosted mode (streamable HTTP)

By default fitter_mcp talks stdio. Pass --http to serve the streamable HTTP transport instead — for a shared team server, a container, or any remote deployment:

# serve MCP at http://<host>:8080/mcp (health probe at /healthz)
FITTER_MCP_AUTH_TOKEN=my-secret fitter_mcp --http :8080

# register the remote endpoint in Claude Code
claude mcp add --transport http fitter http://localhost:8080/mcp --header "Authorization: Bearer my-secret"

• --http <addr> (env FITTER_MCP_HTTP_ADDR) — listen address; stdio mode when empty
• FITTER_MCP_AUTH_TOKEN — when set, every /mcp request must send Authorization: Bearer <token>; without it the endpoint is unauthenticated, so bind to localhost or put it behind a proxy
• --stateless (env FITTER_MCP_STATELESS=true) — no per-session state, so replicas can sit behind a load balancer without sticky sessions

The server shuts down gracefully on SIGINT/SIGTERM.

Docker

A slim multi-arch image (linux/amd64 + linux/arm64) ships with every release:

# hosted HTTP mode
docker run --rm -p 8080:8080 \
  -e FITTER_MCP_HTTP_ADDR=:8080 \
  -e FITTER_MCP_AUTH_TOKEN=my-secret \
  ghcr.io/pxyup/fitter-mcp:latest

# or stdio mode, spawned by the MCP client
claude mcp add fitter -s user -- docker run --rm -i ghcr.io/pxyup/fitter-mcp:latest

The image contains only the fitter binary and CA certificates: server/static/file connectors work, browser connectors (chromium/docker/playwright) do not — use a release binary on a host with a browser for those.

Environment variables

1. FITTER_PLUGINS - string[""] - path for plugins folder, same as the --plugins flag of Fitter/Fitter_CLI
2. FITTER_MCP_HTTP_ADDR - string[""] - listen address for remote mode, same as --http
3. FITTER_MCP_AUTH_TOKEN - string[""] - bearer token protecting the HTTP endpoint
4. FITTER_MCP_STATELESS - bool[false] - stateless HTTP transport, same as --stateless

Recipes

Complete, tested configs showing the main patterns. All of them run unchanged via Fitter_MCP (fitter_run_file), Fitter_CLI or the library — more in examples/.

Scrape a page that has no API, enrich from one that does

GitHub trending has no official API — scrape the HTML for repo slugs (html_attribute reads the href), then fan each one out into the GitHub REST API with {PL}:

examples/config_github_trending.json

{
  "item": {
    "connector_config": {
      "response_type": "HTML",
      "url": "https://github.com/trending",
      "server_config": { "method": "GET", "headers": { "User-Agent": "Mozilla/5.0 (fitter demo)" } }
    },
    "model": {
      "array_config": {
        "root_path": "article.Box-row h2 a",
        "length_limit": 5,
        "item_config": {
          "field": {
            "type": "string",
            "html_attribute": "href",
            "generated": { "model": {
              "type": "object",
              "connector_config": {
                "response_type": "json",
                "url": "https://api.github.com/repos{PL}",
                "server_config": { "method": "GET", "headers": { "User-Agent": "fitter-demo" } },
                "null_on_error": true
              },
              "model": { "object_config": { "fields": {
                "repo": { "base_field": { "type": "string", "path": "full_name" } },
                "stars": { "base_field": { "type": "int", "path": "stargazers_count" } },
                "language": { "base_field": { "type": "string", "path": "language" } }
              } } }
            } }
          }
        }
      }
    }
  },
  "limits": { "host_request_limiter": { "api.github.com": 2 } }
}

[{"repo": "block/buzz", "stars": 6214, "language": "Rust"}, {"repo": "koala73/worldmonitor", "stars": 71179, "language": "TypeScript"}]

Join on a JSON field with an expression

When array items are objects, the join key lives inside them — pull it out with {{{FromExp=...}}} (expr-lang over fRes, the current item). Book search → author details, search query supplied via input:

examples/config_book_authors.json

"url": "https://openlibrary.org/authors/{{{FromExp=fromJSON(fRes).author_key[0]}}}.json"

./fitter_cli --path=examples/config_book_authors.json --input=dune

[{"title": "Dune", "year": 1965, "author": {"name": "Frank Herbert", "born": "8 Octo