by jackulau
Open-source, local-first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost.
Available after the catalog security scan passes.
⚠️ Third-Party Software Notice
This skill is third-party open-source software developed and hosted independently on GitHub. SkillsLLM is an informational directory and does not control or maintain the underlying repository.
Any security checks, ratings, or warnings displayed by SkillsLLM are automated and limited in scope. They do not constitute a security certification or guarantee that the software is safe, error-free, or free from malicious code, vulnerabilities, compromised dependencies, or prompt-injection risks.
Review the source code, permissions, dependencies, and configuration before installing or running any third-party skill. Use is at your own risk. To the maximum extent permitted by applicable law, SkillsLLM is not liable for losses arising from third-party software.
The deep catalog scan for this skill is still queued. Run an instant dependency check now instead.
# Add to your Claude Code skills
git clone https://github.com/jackulau/coding.shGuides for using ai agents skills like coding.sh.
See how coding.sh compares with popular alternatives.
npx coding.sh # try it: finds your agents, asks once, imports past sessions
curl -fsSL https://coding.sh/install | sh # or install it: same setup, then opens the viewer
Run npx coding.sh again to open the viewer on your sessions.
What it costs: nothing. MIT licensed, runs on your machine, no account.
Each session goes into a SQLite index on your machine, with its latency and tokens. You browse it in a local viewer, query it from the CLI or from your agents over MCP, turn failures into evals, and compare changes with paired statistics. No session data leaves your machine unless you add a remote sink.
No runtime dependencies. Node 22.16 or newer.
npx coding.sh, or the install command above. It runs coding.sh init: it finds your
agents, shows one plan and asks once.coding.sh to open the viewer. New sessions show up on their own, with no ingest step, and
your agents can read them over MCP.init points each agent's hooks at coding.sh, registers the MCP server where the agent supports
it, and backs up every config file it edits. It also imports the sessions your agents already keep
on this machine, so the viewer opens on your own history.
| Command | What it does |
|---|---|
coding.sh init --dry-run |
Prints the plan without writing anything. --diff adds the edits. |
coding.sh init --no-import |
Sets up capture and skips the past-session import. |
coding.sh ingest --backfill |
Imports past sessions again, for example after adding an agent. |
coding.sh uninstall |
Removes the hooks and MCP entries again. |
You can also install with npm: npm install -g coding.sh. If you only ran npx coding.sh, the
hooks keep recording, but the MCP server runs the copy in the npx cache, so install it before you
clear that cache. Installing from source, updates and uninstall:
docs/guides/install.md.
| Area | What it does | Start with |
|---|---|---|
| Record | Captures each session, including subagents, from hooks and transcripts. Wraps any other agent. Takes OpenTelemetry from frameworks and apps. | coding.sh init, coding.sh run -- <cmd>, coding.sh collector |
| Measure | Turns, tool calls, model calls, tokens, latency and outcome for every session. Flags sessions that edited code and never ran a check. | coding.sh sessions, coding.sh usage, coding.sh improve |
| Evaluate | Turns a failed session into a replayable eval. Runs experiments with code graders or calibrated LLM judges. A/B tests prompts, harness files and models with paired statistics. | coding.sh eval new, coding.sh exp compare |
| Benchmark | Draws new cases every day, runs every contender on the same frozen set, and writes a static report. | coding.sh bench |
| Production | One server for ingest, the viewer and the API, backed by SQLite, Postgres or ClickHouse. SDKs for API apps. | coding.sh serve |
coding.sh sessions # recent sessions
coding.sh show latest # timeline with turns, tools, subagents and latency
coding.sh search "rate limit" # full-text search over prompts, replies and tool calls
coding.sh usage --by model # tokens, turns and cache hits
coding.sh stats --by agent # sessions, failures, cost and latency per agent, with a daily trend
coding.sh diff-calls latest#-1 # what changed since the previous model call, and why the cache missed
coding.sh ui # the local viewer
Every command takes --json. coding.sh mcp gives your agents the same data as MCP tools.
More in docs/guides/explore.md.
coding.sh run -- <cmd> wraps any other agent and snapshots the repo before and after.coding.sh collector takes OpenTelemetry from agent frameworks and apps.coding.sh init --proxy routes Claude Code and Codex through a local API proxy that logs each
model call's system prompt, tools, messages and cache use. More in
docs/how-to/capture-with-proxy.md.Turn a failed session into a replayable eval with coding.sh eval new, and check that its
verifier is fair with coding.sh eval validate. Run experiments over versioned datasets with code
graders or calibrated LLM judges. coding.sh exp compare and coding.sh compare A/B prompts,
harness files and models with paired statistics. coding.sh exp replay resends one logged model
call with another model or system prompt. Tool evals and multi-agent workflow runs are covered
too. More in docs/guides/evals.md.
coding.sh bench draws new cases every day from a script, yo
coding.sh is an open-source ai agents skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by jackulau. Open-source, local-first observability and evals for AI agents: see what Claude Code, Codex, Cursor and others did and what it cost. It has 68 GitHub stars.
coding.sh's catalog security scan is still queued. You can run an instant dependency and prompt-injection check now with the "Scan for vulnerabilities" button above.
Clone the repository with "git clone https://github.com/jackulau/coding.sh" and add it to your Claude Code skills directory (see the Installation section above).
coding.sh is primarily written in TypeScript. It is open-source under jackulau on GitHub, so you can review or fork the full source.
Yes. SkillsLLM lists many other AI Agents skills you can browse and compare side by side. Open the AI Agents category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh coding.sh against similar tools.
No comments yet. Be the first to share your thoughts!