MCP Local RAG

Search private documents from an MCP client or the terminal without sending them to an embedding API.

mcp-local-rag indexes PDF, DOCX, Markdown, and text files on your machine. Search combines semantic similarity with keyword matching, so queries can match both intent and exact technical terms such as API names, class names, and error codes. Results include source passages and, where available, headings, line numbers, or page numbers so you can check and cite the original document.

No API key, Docker, Python, or external database is required. After the initial model download, text ingestion and search work offline.

Quick Start

Requirements

• Node.js 22 or later
• Internet access on first use to download the npm package and embedding model
• A directory containing the documents you want to search

Set BASE_DIR to that directory. It is also the security boundary for file operations. Replace /absolute/path/to/your/documents below with the directory's absolute path.

Use one of the examples below, or register npx -y mcp-local-rag and set BASE_DIR using your client's MCP configuration format.

Set DB_PATH and CACHE_DIR to absolute paths as well. Relative paths resolve from the server's working directory, so starting the server from different projects creates a separate index and model cache in each.

Run this command:

claude mcp add local-rag --scope user --env BASE_DIR=/absolute/path/to/your/documents -- npx -y mcp-local-rag

Add to ~/.codex/config.toml:

[mcp_servers.local-rag]
command = "npx"
args = ["-y", "mcp-local-rag"]

[mcp_servers.local-rag.env]
BASE_DIR = "/absolute/path/to/your/documents"

Add to ~/.config/opencode/opencode.json (or opencode.jsonc):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "local-rag": {
      "type": "local",
      "command": ["npx", "-y", "mcp-local-rag"],
      "environment": {
        "BASE_DIR": "/absolute/path/to/your/documents"
      }
    }
  }
}

Add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "local-rag": {
      "command": "npx",
      "args": ["-y", "mcp-local-rag"],
      "env": {
        "BASE_DIR": "/absolute/path/to/your/documents"
      }
    }
  }
}

Restart the client, then ask it to build the index:

Sync all documents in the configured root and wait until it finishes.

The first sync downloads the default embedding model (about 90 MB) and may take 1–2 minutes before ingestion starts. Later runs use the local cache.

Once the sync completes:

What does the API documentation say about authentication?

CLI Quick Start

To use the CLI without an MCP client:

npx mcp-local-rag ingest ./docs/
npx mcp-local-rag query "authentication API"

The CLI uses the current directory as its document root by default. Run both commands from the same directory so they use the same default index, or set BASE_DIR and DB_PATH explicitly.

Supported Content

HTML fetching is not built into the server. An MCP client can fetch a page and pass its HTML to ingest_data.

Excel, PowerPoint, standalone images, and source-code file extensions are not supported by file ingestion. PDFs can optionally use a local vision model to describe figures, but this is not OCR or image search.

Using the Index

Sync after adding, editing, or removing documents. For searches and follow-up reading, ask your MCP client:

Find the documented behavior of ERR_CONNECTION_REFUSED.
Read the surrounding chunks for that result.

You can also ingest a single file or HTML already fetched by the client. To refresh an existing entry, ingest the document again using the same path or source identifier. sync skips unchanged files. MCP file paths must be absolute and inside a configured document root.

PDF section headings may be inaccurate. Check the original page when you need the exact heading.

CLI

Use the CLI to update the index, narrow searches, or remove indexed content:

npx mcp-local-rag sync ./docs/
npx mcp-local-rag query "auth" --scope /docs/api --scope /docs/guide
npx mcp-local-rag read-neighbors --file-path /abs/path.md --chunk-index 5
npx mcp-local-rag list
npx mcp-local-rag status
npx mcp-local-rag delete ./docs/old.pdf
npx mcp-local-rag delete --source "https://example.com/docs"

ingest imports the selected files; sync also removes entries for deleted files and skips unchanged files. Use --scope to restrict search results to a path prefix, repeating it to include multiple prefixes.

Global options such as --db-path, --cache-dir, and --model-name go before the subcommand. Subcommand options go after it:

npx mcp-local-rag --db-path ./my-db query "authentication"

Run npx mcp-local-rag --help for the complete command reference.

query writes its results to stdout as JSON, best match first, so it can be piped into another tool. The field-by-field contract is in docs/schema/query-output.schema.json.

Agent Skills

Agent Skills provide query and ingestion guidance for AI assistants:

npx mcp-local-rag skills install --claude-code
npx mcp-local-rag skills install --claude-code --global
npx mcp-local-rag skills install --codex

Installed skills cover query formulation, result refinement, and HTML ingestion. Ask the assistant to use the mcp-local-rag skill explicitly if it does not activate automatically.

Advanced Options

Start with the defaults. Open the sections below when you need different document roots, better results for your corpus, or searchable PDF figures.

The MCP server reads environment variables. The CLI accepts the listed variables and flags. Keep the same DB_PATH when commands should use the same index.

File operations stay within configured roots. For multiple directories, set BASE_DIRS='["/absolute/docs","/absolute/specs"]' or repeat CLI --base-dir. Precedence: CLI roots, BASE_DIRS, BASE_DIR, then the current directory. Only the highest-priority source is used; roots from different sources are not merged. Invalid BASE_DIRS is an error. Relative DB_PATH and CACHE_DIR are resolved from the working directory.

Choose an embedding model for your documents’ language and subject. Compare settings using questions you actually ask and check which source passages are returned. The model must support mean pooling and L2 normalization, which this tool uses to produce embeddings.