MCP Local RAG

Local RAG for developers via MCP or CLI. Semantic search with keyword boost for exact technical terms — fully private, zero setup.

Features

• Semantic search with keyword boost Vector search first, then keyword matching boosts exact matches. Terms like useEffect, error codes, and class names rank higher—not just semantically guessed.

• Smart semantic chunking Chunks documents by meaning, not character count. Uses embedding similarity to find natural topic boundaries—keeping related content together and splitting where topics change.

• Quality-first result filtering Groups results by relevance gaps instead of arbitrary top-K cutoffs. Get fewer but more trustworthy chunks.

• Runs entirely locally No API keys, no cloud, no data leaving your machine. Works fully offline after the first model download.

• Zero-friction setup One npx command. No Docker, no Python, no servers to manage. Use via MCP, CLI, or both. Optional Agent Skills help AI assistants form better queries and interpret results.

Quick Start

Set BASE_DIR to the folder you want to search (or BASE_DIRS for multiple roots — see Configuration). Documents must live under one of the configured roots.

Add the MCP server to your AI coding tool:

For Cursor — Add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "local-rag": {
      "command": "npx",
      "args": ["-y", "mcp-local-rag"],
      "env": {
        "BASE_DIR": "/path/to/your/documents"
      }
    }
  }
}

For Codex — Add to ~/.codex/config.toml:

[mcp_servers.local-rag]
command = "npx"
args = ["-y", "mcp-local-rag"]

[mcp_servers.local-rag.env]
BASE_DIR = "/path/to/your/documents"

For Claude Code — Run this command:

claude mcp add local-rag --scope user --env BASE_DIR=/path/to/your/documents -- npx -y mcp-local-rag

Restart your tool, then start using it:

You: "Ingest api-spec.pdf"
Assistant: Successfully ingested api-spec.pdf (47 chunks created)

You: "What does the API documentation say about authentication?"
Assistant: Based on the documentation, authentication uses OAuth 2.0 with JWT tokens.
          The flow is described in section 3.2...

Or use directly as CLI — no MCP server needed:

npx mcp-local-rag ingest ./docs/
npx mcp-local-rag query "authentication API"

That's it. No Docker, no Python, no server setup.

Why This Exists

You want AI to search your documents—technical specs, research papers, internal docs. But most solutions send your files to external APIs.

Privacy. Your documents might contain sensitive data. This runs entirely locally.

Cost. External embedding APIs charge per use. This is free after the initial model download.

Offline. Works without internet after setup.

Code search. Pure semantic search misses exact terms like useEffect or ERR_CONNECTION_REFUSED. Keyword boost catches both meaning and exact matches.

Agent reality. In practice, many AI environments mainly use tool calling. CLI support and Agent Skills make the same workflows available even without full MCP integration.

Usage

mcp-local-rag provides two interfaces: an MCP server for AI coding tools and a CLI for direct use from the terminal.

Using with MCP

The MCP server provides 7 tools: ingest_file, ingest_data, query_documents, read_chunk_neighbors, list_files, delete_file, status.

Ingesting Documents

"Ingest the document at /Users/me/docs/api-spec.pdf"

Supports PDF, DOCX, TXT, and Markdown. The server extracts text, splits it into chunks, generates embeddings locally, and stores everything in a local vector database.

Re-ingesting the same file replaces the old version automatically.

Ingesting PDFs with figures (visual mode)

PDFs with charts, tables, or diagrams can optionally add local VLM-generated captions to the document index, giving visual content some searchable representation in the same vector + FTS pipeline. Captions are auxiliary text — not image search, not OCR, and not a faithful transcription of the figure.

Via MCP:

"Ingest /Users/me/docs/api-spec.pdf with visual: true"

Via CLI:

npx mcp-local-rag ingest ./docs/spec.pdf --visual

Each caption is emitted as its own chunk with the envelope [Visual content on page N: …], alongside the page-body chunks. It flows through the existing embedder and FTS index — no schema differences, no separate index.

Visual mode is opt-in; normal ingest does not load the VLM. Per-page VLM failures are tolerated — that page proceeds with text only.

Choosing a visual-quality profile

Visual mode offers two profiles, selected per ingest call:

| Profile | Model | Disk (cache) | Per-page inference | Suited for | |---|---|---|---|---| | fast (default) | HuggingFaceTB/SmolVLM-256M-Instruct | ~250 MB | baseline | Light visual indexing, quick first-run setup. | | quality | onnx-community/Qwen2.5-VL-3B-Instruct-ONNX | ~2.9 GB | ~2× fast | Figures with in-image text (axis labels, panel sub-labels, annotations) where caption fidelity matters more than inference time. |

The numbers above are measured on CPU during development on the project's probe PDFs; they may shift with model updates or differ on your hardware.

Via MCP — ingest_file accepts an optional visualQuality parameter (enum: 'fast' | 'quality', default 'fast'; ignored when visual is false):

"Ingest /Users/me/docs/research-paper.pdf with visual: true and visualQuality: 'quality'"

Via CLI — --visual-quality fast|quality (default fast; silently ignored when --visual is absent):

npx mcp-local-rag ingest ./docs/research-paper.pdf --visual --visual-quality quality

Profile model identifiers and quantization variants are fixed per release. Both profiles share the same CACHE_DIR (default: ./models/); the first run on each profile downloads its model.

Behavior change from v0.14.0: Captions are now emitted as dedicated chunks rather than appended to the page text before chunking. As a side effect, metadata.fileSize for visual ingests no longer includes the caption character count — it measures the post-extraction body length only. The underlying PDF is unchanged; only the reported fileSize for visual-ingested PDFs may shrink across the release boundary.

Security note: Visual captions are derived from PDF contents and may inherit attacker-controlled text. Downstream LLM consumers should treat retrieved chunks as untrusted data, not as instructions. The [Visual content on page N: …] envelope helps consumers distinguish caption text from prose.

Ingesting HTML Content

Use ingest_data to ingest HTML content retrieved by your AI assistant (via web fetch, curl, browser tools, etc.):

"Fetch https://example.com/docs and ingest the HTML"

The server extracts main content using Readability (removes navigation, ads, etc.), converts to Markdown, and indexes it. Perfect for:

• Web documentation
• HTML retrieved by the AI assistant
• Clipboard content

HTML is automatically cleaned—you get the article content, not the boilerplate.

Note: The RAG server itself doesn't fetch web content—your AI assistant retrieves it and passes the HTML to ingest_data. This keeps the server fully local while letting you index any content your assistant can access. Please respect website terms of service and copyright when ingesting external content.

Searching Documents

"What does the API documentation say about authentication?"
"Find information about rate limiting"
"Search for error handling best practices"

Search uses semantic similarity with keyword boost. This means useEffect finds documents containing that exact term, not just semantically similar React concepts.

Results include text content, source file, document title, and relevance score. The document title provides context for each chunk, helping identify which document a result belongs to. Adjust result count with limit (1-20, default 10).

Expanding Context Around a Result

When a search result needs more surrounding context, use read_chunk_neighbors to read the chunks before and after it:

"That result about authentication looks relevant — read the surrounding chunks for the full explanation"

Pass the filePath and chunkIndex from the search result. The response includes the target chunk (marked isTarget: true) plus its neighbors, sorted by chunk index. Defaults to 2 chunks before and 2 after (adjustable up to 50 each).

Managing Files

"List all files in configured base directories and their ingested status"   # See what's indexed
"Delete old-spec.pdf from RAG"     # Remove a file
"Show RAG server status"           # Check system health

Using as CLI

All MCP tools are also available as CLI commands — no MCP server needed:

npx mcp-local-rag ingest ./docs/               # Bulk ingest files
npx mcp-local-rag query "authentication API"    # Search documents
npx mcp-local-rag read-neighbors --file-path /abs/path.md --chunk-index 5  # Expand context
npx mcp-local-rag list                          # Show ingestion status
npx m