tokdiet

Your AI agent is paying to send the same file dump five times. tokdiet is a local proxy that sits between your agent and the model API, meters every token, puts your bloated context on a diet — and proves the answer didn't get worse.

ccusage that shrinks the bill — without losing quality.

🌐 Live demo (watch one request lose the weight): agiwhitelist.github.io/tokdiet 📝 Launch write-up + full benchmark methodology: I cut an AI agent's input tokens by 71% and quality held — here's the 66-task benchmark

The proof (this is the whole point)

Every "context optimizer" cuts tokens. The scary question is the one they can't answer:

"If I cut the context, does the model get dumber?"

So we measured it. A 66-task A/B benchmark across 6 categories on a real model (MiniMax‑M3), each task run twice — full context (baseline) vs through tokdiet (governed) — graded against the known answer, repeated ×3 and majority‑voted to cancel model noise:

                       baseline      tokdiet
  input tokens          5.07M    →    1.46M       −71%
  quality (66 tasks)     64/66        63/66        ≈ parity (95–97%)
  ─────────────────────────────────────────────────────────
  198 paired runs · LLM-judge 92% similarity · confirmed on a 2nd model (MiniMax-M2.5: −72%)

−71% tokens, quality on par with baseline. Real requests, real grading — not a mock. The ~1–2 task gap is model nondeterminism plus the model declining to echo a secret — not context loss; the hardest "needle buried in junk" adversarial cases pass, because tokdiet doesn't delete blindly — it pages cold context out recoverably and protects anything on‑topic. Reproduce it yourself: node bench/run.mjs (needs an API key in env).

How it compares

Everyone shows the bill or cuts it blind. tokdiet is the one that cuts it and proves the model didn't get dumber — and stops cutting the moment it might.

Quick start

# 1. Start the proxy (and live dashboard) — no install needed
npx tokdiet start

# 2. Point your agent at the proxy instead of the real API
export ANTHROPIC_BASE_URL=http://localhost:7787
export OPENAI_BASE_URL=http://localhost:7787/v1

Now run your agent (Claude Code, Cursor, Codex, your own script) as usual. Traffic flows through tokdiet, gets metered and compacted, and is forwarded upstream unchanged in every way that matters.

Your API key stays with you. tokdiet reads x-api-key / Authorization only to forward them upstream. They are never written to SQLite and never written to any log. And it's fail‑open: if anything inside the governor errors, it falls back to transparent passthrough — the proxy will never break your request or surface its own 5xx.

Default ports: proxy 7787, dashboard 7878. Override with --port / --dashboard-port.

Install via Claude Code

tokdiet ships as a Claude Code plugin via its own marketplace:

/plugin marketplace add agiwhitelist/tokdiet
/plugin install tokdiet

What the plugin does — and what it doesn't. The plugin ships a lightweight metering hook plus a /tokdiet command. The hook runs on every tool call (PreToolUse + PostToolUse) and logs tool I/O byte sizes to ~/.tokdiet/tool-meter.log. It does not save tokens by itself — a plugin can't set ANTHROPIC_BASE_URL for the Claude Code process, so it can't route your traffic through the compacting proxy.

The actual token savings come from the proxy. Start it and point Claude Code at it (this is what gives you the ~−71% token reduction):

npx tokdiet start
export ANTHROPIC_BASE_URL=http://localhost:7787   # then launch Claude Code from this shell

View metered tokens, cost, and savings any time with npx tokdiet report, or run /tokdiet inside Claude Code for these instructions.

Works with Claude Code (and it's careful about it)

Claude Code is the flagship use case, and it has two landmines a naive compacting proxy walks straight into. tokdiet handles both:

• Prompt caching. Claude Code marks a cached prefix with cache_control; cached input costs ~10% of normal. Rewriting that prefix invalidates the cache and can make a request cost more. tokdiet is cache‑aware — it never touches content at or before a cache_control breakpoint.
• Extended thinking. Claude Code sends signed thinking blocks that Anthropic requires returned verbatim; touching one is an instant 400. tokdiet is thinking‑safe — signed/thinking blocks are never surfaced or mutated.

Both are covered by regression tests (tests/cc-compat.test.ts).

A note on honesty: the dollar‑savings story applies to pay‑per‑token API keys (MiniMax, Anthropic API, OpenAI, …). On a flat Claude subscription there are no per‑token charges to cut, so the value there is metering, budgets, and the live dashboard — not dollars.

How it works

tokdiet is a streaming reverse proxy. SSE responses are proxied incrementally (never buffered whole), so your agent's tokens still stream in real time.

                            tokdiet (localhost:7787)
   agent  ─────────────────────────────────────────────────────────────►  model API
 (Claude  request    ┌───────────┐  ┌───────┐  ┌────────┐  ┌───────────┐   (Anthropic /
  Code,  ──────────► │interceptor│─►│ meter │─►│ budget │─►│ compactor │──►   OpenAI /
  Cursor, raw key    └───────────┘  └───────┘  └────────┘  └─────┬─────┘      Gemini /
  Codex,  forwarded   detect          count      session/        │ dedup / elision /  MiniMax)
  …)                  provider,       tokens     day / repo      │ mid-summarize
                      keep body        & cost     limits          ▼
                      byte-faithful                          ┌───────────────┐
   response                                                  │ quality guard │
 ◄──────────────────────────────────────────────────────────┤ shadow-eval + │
   streamed back, token-for-token                            │  safe-mode    │
                                          ┌──────────────┐   └───────┬───────┘
                                          │ store(SQLite)│◄──────────┘
                                          │ + dashboard  │  telemetry, savings, degradation
                                          └──────────────┘

Context as virtual memory (the idea)

Blind compaction is "delete and pray." tokdiet treats your context like virtual memory: hot content (recent, pinned, relevant to the current question) stays resident; cold content (stale, redundant) is paged out to a local store as a recoverable stub — not deleted. The full block is kept in SQLite keyed by an id, so it can be audited and (roadmap) paged back in on demand when the model actually needs it.

The 3 quality mechanisms

Compaction strategies (safest‑first)

1. Dedup — loss‑free. When the same large block is re‑pasted across a conversation, keep the freshest copy verbatim and replace earlier copies with a pointer marker. Works on near‑duplicates too (a file re‑pasted with a few lines changed), not just byte‑identical ones.
2. Elision — recoverable. Page out the bulk of old tool results (file dumps, command output), keeping a preview plus the salient lines (errors, ids, KEY=VALUE, URLs, paths, numbers) and storing the full body for recovery. Recent, pinned, and question‑relevant results are kept intact.
3. Mid‑summarize (off by default) — summarize mid‑history with a cheap model. Opt‑in (it costs money).

Commands

tokdiet <command> [flags]   # alias: td