stuntd: a local Jev-compatible proxy that learns your typed LLM decisions

stuntd is a local, self-hosted proxy that records the typed decisions your app already makes and learns to answer them itself. It speaks the Jev System One protocol (POST /v1/systemone, with choice, score and noul questions), the OpenAI Chat Completions API and the Anthropic Messages API, distils each decision site into a small head on a frozen Laya encoder, and serves that answer locally with calibrated confidence, handing anything it is unsure about back to the provider.

Try it in the browser: the Hugging Face Space puts zero-shot Laya and the heads stuntd trained side by side on three demos, no install.

Three ways to run it:

• Local Jev. No key, no provider, no training. Point typesafe-sdk at stuntd and the base Laya checkpoint answers your typed questions.
• In front of OpenAI or Anthropic. A drop-in replacement for the provider base URL. Typed requests are recorded; everything else is relayed byte for byte.
• In front of the paid Jev API. stuntd relays, learns from the provider's own answers, and takes the decision over once its head is right often enough.

Status: early release, v0.1. Every number below traces to a demo in this repository or to a source listed at the end.

Who it is for

• You have no Jev key. TypeSafe's Jev is an early-access API at the time of writing, and there are no published weights. stuntd answers the same typed questions locally on Laya, with no key and no training, so a typesafe-sdk client keeps working while you wait.
• You pay a provider for the same decision thousands of times a day. Routing, triage, moderation, gating. stuntd distils those answers into a local head: tens of milliseconds, no tokens, no round trip.
• Zero-shot Laya is not good enough on your labels. On banking77's 77 intents it scores 38.2 where Jev scores 76.4. A few thousand rows of your own traffic is what closes that gap: in the banking demo the risk question goes from 30.0% to 72.5%.
• You want a Claude Code hook that decides locally. A PreToolUse gate answering "may this command run" in about 50 ms, without sending every command you type to a provider. examples/devtools/ ships the hook.

Quickstart: local Jev without an API key

pip install "stuntd[train]"
stuntd config init
stuntd serve

config init writes a commented stuntd.toml with everything at its default, which means no OpenAI upstream and no Jev provider. serve says so and loads the base checkpoint:

stuntd listening on http://127.0.0.1:8787
no upstream: only the Jev routes are served
serving jev locally with convaiinnovations/laya

Then change the base URL in your client and nothing else:

from typesafe_sdk import Choice, Noul, TypeSafeClient

QUESTIONS = {
    "category": Choice(
        instructions="Which part of the product is this ticket about?",
        criteria={"billing": "Payments and refunds.", "bug": "Something is broken."},
    ),
    "needs_human": Noul(instructions="Does a support agent have to read this?"),
}

with TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8787") as client:
    answer = client.system_one(
        state={"subject": "Charged twice", "body": "Two identical charges on Friday."},
        questions=QUESTIONS,
    )
print(answer.choices["category"].choice, answer.nouls["needs_human"].noul)

Anything that speaks the protocol plugs in the same way, by changing one base URL wherever the client exposes it: typesafe-sdk, JevRouter, the local-server request of fast-jev-compaction, and the browser and game agents built on the same SDK.

The key is ignored unless you set [jev] require_key = true. To train a head on this daemon you need labelled rows: either put the paid Jev API in front of it ([jev] upstream) and let it teach, or write the rows yourself and stuntd import them.

Quickstart: OpenAI-compatible proxy that learns

pip install "stuntd[train]"
stuntd config init
stuntd serve --upstream https://api.openai.com

--upstream is the provider origin only, without /v1: your client's own path is appended to it unchanged. In the client, point base_url at the daemon:

import os

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8787/v1", api_key=os.environ["OPENAI_API_KEY"])

Everything keeps working exactly as before. A request whose response_format (or single tool) asks for an object made only of enum, boolean or number fields is a typed decision, and stuntd records it with the provider's answer. Anything else is relayed untouched, streaming included.

An object with several fields, such as {category, urgency, needs_human}, learns one head per field. Each field is its own site, named <site>.<field>, and a request is answered locally only when every field's head is live and confident. If one is not, the provider answers and every field is recorded.

Once a site has a few hundred captures:

stuntd status                  # what has been recorded, per site
stuntd train                   # one head per site with enough examples
stuntd report                  # holdout agreement, ECE, the operating point
stuntd enable <site>           # let that site answer locally

Quickstart: in front of Anthropic

pip install "stuntd[train]"
stuntd config init
stuntd serve --upstream https://api.anthropic.com

Point the SDK at the daemon; the key and the anthropic-version header are forwarded as they are:

import anthropic

client = anthropic.Anthropic(base_url="http://127.0.0.1:8787")

A non-streaming POST /v1/messages that asks for a JSON schema through output_config.format, or for a single tool with an input_schema, is a typed decision on the same terms as the OpenAI path. The same decision asked through either provider lands on the same site and shares one head. A streaming request, or a request with tools and a free-text reply, is relayed untouched. Claude Code and the Agent SDK reach the daemon through ANTHROPIC_BASE_URL.

The local answer is checked against the anthropic SDK's Message model in the tests, but no live Anthropic key has been run through stuntd.

Numbers

What a trained head does, measured on a laptop RTX 5060 with convaiinnovations/laya as the base checkpoint, 3000 generated rows per site and 24 epochs (5890 rows for Snake), with training.cache_encoder at its default. "Zero-shot" is the same daemon before training, answering from the base checkpoint alone.

Every decision site of the three text demos, measured against its teacher on held-out requests before and after the head was trained:

Snake is scored by what the answers do rather than by how many of them are right:

Five of the seven sites in the three text demos reach a holdout agreement of 0.95 or better. The demos run at the default target_agreement of 0.99, which picks each head's operating point rather than being a score it reached. The two that fall short, support's urgency at 0.935 and banking's risk at 0.937, stay in the demos anyway: see limitations.

Latency, size and cost:

The first four rows were measured during development on an RTX 5060 laptop running Windows. They come from no published benchmark, and there is no link to give for them. The two training rows are the Snake and support demos of this repository, on the same laptop with the encoder cache on; the devtools demo measured the same 24-epoch run at 730 s with the cache off against 231 s with it on, 3.2x.

Why train at all, rather than run Laya zero-shot. From jevbench, n=500 per dataset, accuracy:

Zero-shot Lay