stuntd: a local Jev-compatible proxy that learns your typed LLM decisions

stuntd is a local, self-hosted proxy that records the typed decisions your app already makes and learns to answer them itself. It speaks the Jev System One protocol (POST /v1/systemone, with choice, score and noul questions), the OpenAI Chat Completions API and the Anthropic Messages API, distils each decision site into a small head on a frozen Laya encoder, and serves that answer locally with calibrated confidence, handing anything it is unsure about back to the provider.

Try it in the browser: the Hugging Face Space puts zero-shot Laya and the heads stuntd trained side by side on three demos, no install.

Three ways to run it:

• Local Jev. No key, no provider, no training. Point typesafe-sdk at stuntd and the base Laya checkpoint answers your typed questions.
• In front of OpenAI or Anthropic. A drop-in replacement for the provider base URL. Typed requests are recorded; everything else is relayed byte for byte.
• In front of the paid Jev API. stuntd relays, learns from the provider's own answers, and takes the decision over once its head is right often enough.

Status: early release, v0.1.3. Every number below traces to a demo in this repository or to a source listed at the end.

Who it is for

• You have no Jev key. TypeSafe's Jev is an early-access API at the time of writing, and there are no published weights. stuntd answers the same typed questions locally on Laya, with no key and no training, so a typesafe-sdk client keeps working while you wait.
• You pay a provider for the same decision thousands of times a day. Routing, triage, moderation, gating. stuntd distils those answers into a local head: tens of milliseconds, no tokens, no round trip.
• Zero-shot Laya is not good enough on your labels. On banking77's 77 intents it scores 38.2 where Jev scores 76.4. A few thousand rows of your own traffic is what closes that gap: in the banking demo the risk question goes from 30.0% to 72.5%.
• You want a Claude Code hook that decides locally. A PreToolUse gate answering "may this command run" in about 50 ms, without sending every command you type to a provider. examples/devtools/ ships the hook.

Quickstart: local Jev without an API key

pip install "stuntd[train,jev]"
stuntd config init
stuntd serve

config init writes a commented stuntd.toml with everything at its default, which means no OpenAI upstream and no Jev provider. serve lists the endpoints it serves and loads the base checkpoint:

stuntd listening on http://127.0.0.1:8787
endpoints: jev local
serving jev locally with convaiinnovations/laya

stuntd stop ends the daemon, and GET /healthz answers 200 with the number of sites in each mode without loading a checkpoint.

Then change the base URL in your client and nothing else:

from typesafe_sdk import Choice, Noul, TypeSafeClient

QUESTIONS = {
    "category": Choice(
        instructions="Which part of the product is this ticket about?",
        criteria={"billing": "Payments and refunds.", "bug": "Something is broken."},
    ),
    "needs_human": Noul(instructions="Does a support agent have to read this?"),
}

with TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8787") as client:
    answer = client.system_one(
        state={"subject": "Charged twice", "body": "Two identical charges on Friday."},
        questions=QUESTIONS,
    )
print(answer.choices["category"].choice, answer.nouls["needs_human"].noul)

Anything that speaks the protocol plugs in the same way, by changing one base URL wherever the client exposes it: typesafe-sdk, JevRouter, the local-server request of fast-jev-compaction, and the browser and game agents built on the same SDK.

The key is ignored unless you set [jev] require_key = true. To train a head on this daemon you need labelled rows: either put the paid Jev API in front of it ([jev] upstream) and let it teach, or write the rows yourself and stuntd import them (see importing your own rows).

Try it without any key

Nothing here needs an account, a provider or a training run: pip install "stuntd[train,jev]", stuntd serve, and run the client above. The first start downloads the Laya checkpoint (see Details worth knowing).

To see how each answer was served, post the questions yourself and read the X-Stuntd header:

curl -si http://127.0.0.1:8787/v1/systemone -H "content-type: application/json" -d '{
  "state": "Charged twice for the same invoice",
  "model": "jev-latest",
  "questions": {"needs_human": {"type": "noul", "instructions": "Does a person have to read this?"}}
}'

With no head trained, the header reads X-Stuntd: jev; mode=local; questions=1; live=0; zeroshot=1: the base checkpoint answered. The demos under examples/ generate their own labelled rows, so stuntd import, stuntd train and stuntd enable can be tried on data with a known right answer before any of your own traffic is involved.

The local fallback

In local Jev mode nothing sits behind the daemon, so a question a head will not answer has to be answered by something. By default that is the base Laya, zero-shot: the head answers when its confidence reaches the site's threshold and the request is not flagged by the novelty gate, and the base checkpoint answers the rest. serving.local_fallback = "head" makes the head answer those too, for a head you trust more than zero-shot on your labels. stuntd enable warns when a site's coverage is under 50% in local mode with the zero-shot fallback, since most answers would then come from the base checkpoint. stuntd report SITE --gold FILE scores what is actually served under whichever fallback is set, and prints which one it used.

Quickstart: OpenAI-compatible proxy that learns

pip install "stuntd[train]"
stuntd config init
stuntd serve --upstream https://api.openai.com

--upstream is the provider origin only, without /v1: your client's own path is appended to it unchanged. In the client, point base_url at the daemon:

import os

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8787/v1", api_key=os.environ["OPENAI_API_KEY"])

Everything keeps working exactly as before. A request whose response_format (or single tool) asks for an object made only of enum, boolean or number fields is a typed decision, and stuntd records it with the provider's answer. Anything else is relayed untouched, streaming included.

An object with several fields, such as {category, urgency, needs_human}, learns one head per field. Each field is its own site, named <site>.<field>, and a request is answered locally only when every field's head is live and confident. If one is not, the provider answers and every field is recorded.

Once a site has a few hundred captures:

stuntd status                  # what has been recorded, per site
stuntd train                   # one head per site with enough examples
stuntd report                  # holdout agreement, ECE, the operating point
stuntd enable <site>           # let that site answer locally

enable and disable also take the parent of a multi-field decision: stuntd enable triage acts on every triage.<field> that has a head and prints each one. While some fields are not live, the live ones are still compared against the provider's answer on every request, so their agreement keeps moving and a field that drifts is demoted on its own.

Quickstart: in front of Anthropic

pip install "stuntd[train]"
stuntd config init
stuntd serve --upstream https://api.anthropic.com

Point the SDK at the daemon; the key and the anthropic-version header are forwarded as they are:

import anthropic

client = anthropic.Anthropic(base_url="http://127.0.0.1:8787")

A non-streaming POST /v1/messages that asks for a JSON schema through output_config.format, or for a single tool with an input_schema, is a typed decision on the same terms as the OpenAI path. The same decision asked through either provider lands on the same site and shares one head. A streaming request, or a request with tools and a free-text reply, is relayed untouched. Claude Code and the Agent SDK reach the daemon through ANTHROPIC_BASE_URL.

The local answer is checked against the anthropic SDK's Message model in the tests, but no live Anthropic key has been run through stuntd.

Numbers

What a trained head does, measured on a laptop RTX 5060 with convaiinnovations/laya as the base checkpoint, 3000 generated rows per site and 24 epochs (5890 rows for Snake), with training.cache_encoder at its default. "Zero-shot" is the same daemon before training, answering from the base checkpoint alone.