Reduce Claude Code, Codex, OpenCode wall clock and token use by 50% with open source, local semantic search. Works for small and large codebases and monorepos! Enterprise-ready and fully compliant via Ollama and SQLite-vec.
AI Agentsagentic-codingclaudeclaude-aiclaude-codeclaude-pl
Installation
# Add to your Claude Code skills
git clone https://github.com/ory/lumen
README.md
Claude reads entire files to find what it needs. Lumen gives it a map.
Lumen is a 100% local semantic code search engine for AI coding agents. No API
keys, no cloud, no external database, just open-source embedding models
(Ollama or LM Studio), SQLite,
and your CPU. A single static binary and your own local embedding server.
The payoff is measurable and reproducible: across 8 benchmark runs on 8
languages and real GitHub bug-fix tasks, Lumen cuts cost in every single
language — up to 39%. Output tokens drop by up to 66%, sessions complete up to
53% faster, and patch quality is maintained in every task. All verified with a
transparent, open-source benchmark framework that you can
run yourself.
Claude Code asking about the
Prometheus codebase. Lumen's
semantic_search finds the relevant code without reading entire files.
Quick start
Prerequisites:
Platform support: Linux, macOS, and Windows. File locking for background
indexing coordination uses flock(2) on Unix and LockFileEx on Windows
(via gofrs/flock).
Ollama installed and running, then pull the default
embedding model:
Git worktree support — worktrees share index data automatically; a new
worktree seeds from a sibling's index and only re-indexes changed files,
turning minutes of embedding into seconds
Zero cloud — embeddings stay on your machine; no data leaves your network
Ollama and LM Studio — works with either local embedding backend
How it works
Lumen sits between your codebase and Claude as an MCP server. When a session
starts, it walks your project and builds a Merkle tree over file hashes:
only changed files get re-chunked and re-embedded. Each file is split into
semantic chunks (functions, types, methods) using Go's native AST or tree-sitter
grammars for other languages. Chunks are embedded and stored in SQLite +
sqlite-vec using cosine-distance KNN for retrieval.
When Claude needs to understand code, it calls semantic_search instead of
reading entire files. The index is stored outside your repo
(~/.local/share/lumen/<hash>/index.db), keyed by project path and model name —
different models never share an index.
Benchmarks
Lumen is evaluated using bench-swe: a SWE-bench-style harness that runs
Claude on real GitHub bug-fix tasks and measures cost, time, output tokens, and
patch quality — with and without Lumen. All results are reproducible: raw JSONL
streams, patch diffs, and judge ratings are committed to this repository.
Key results — 8 runs across 8 languages, hard difficulty, real GitHub
issues (ordis/jina-embeddings-v2-base-code, Ollama):
Cost was reduced in every language tested. Quality was maintained in every
task — zero regressions. JavaScript and TypeScript show the most dramatic
efficiency gains: same quality fixes in half the time with two-thirds fewer
tokens. Even on tasks too hard for either approach (Rust), Lumen cuts the cost
of failure by 39%.
See docs/BENCHMARKS.md for all 8 per-language deep dives,
judge rationales, and reproduce instructions.
Supported languages
Supports 12 language families with semantic chunking (9 benchmarked):
Go uses the native Go AST parser for the most precise chunks. All other
languages use tree-sitter grammars. See docs/BENCHMARKS.md
for all 9 per-language benchmark deep dives.
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
