Open Index

Open Index is a tool for building domain specific accurate, structured data that agents can actually operate on — and for keeping that data correct as things change.

You use Open Index to build a brain: a searchable, continuously-improving context graph of your domain. A brain is domain-agnostic — model a support org (product → "has common issue" → issue), a sales pipeline (customer → order), your infrastructure (service → runbook), or anything else. You define the concepts; Open Index stores them, searches them, and draws the map.

A brain is built from four primitives:

• doc_type — a concept you want to track and maintain (e.g. service, customer, issue).
• doc_schema — the fields stored for a given doc_type.
• entity — one instance of a doc_type, stored per its schema. Every entity can link to others via related_to (the target) + relationship_edge_meaning (free-text edge semantics).
• connector — an optional source you extract entities from (e.g. an MCP server).

Quickstart

pip install -e '.[all]'          # core + explorer UI + MCP server

# Try the bundled example (support brain: products, issues, segments, comments)
open-index index --brain examples/support-brain
open-index ui    --brain examples/support-brain      # open the Map tab, pick an anchor

# Or start your own brain from scratch
open-index init my-brain
open-index add-doc-type customer --brain my-brain
# ...add entities under my-brain/entities/**/*.json...
open-index index --brain my-brain
open-index ui    --brain my-brain

Prefer containers, or need a brain several agents share? → docs/deployment.mdx (docker compose --profile sqlite up).

Commands

A context layer for domain-specialized agents

Open Index is designed to sit behind agents specialized for a domain—legal, marketing, customer support, sales, infrastructure, or a domain of your own. The MCP server gives those agents structured context and a validated way to keep that context current:

• agent prompt — dynamic domain navigation is published through MCP server instructions so supporting hosts can inject it before the first turn
• read — navigation_guidelines() refreshes those instructions; search_brain() and get_entity() retrieve domain context
• write — put_entity() (add/update an entity), create_doc_type() (define a concept)

Read and write is the default MCP mode so a domain agent can both use knowledge and maintain it. Add --read-only when the agent should consume context without mutating it. Claude Code is supported as one optional MCP client; open-index init scaffolds .mcp.json, CLAUDE.md, and an editing skill as conveniences for it.

Portable agent setup skill

skills/setup-open-index/SKILL.md follows the portable Agent Skills SKILL.md format used by agent runtimes including OpenClaw, Hermes, and Claude Code. Give or install this skill in the selected runtime when the agent should set up Open Index itself. It covers installation, domain-brain initialization, generic MCP wiring, default read/write verification, the --read-only opt-out, and production guardrails.

A brain on disk

my-brain/
  brain.yaml            # name + storage/search backend
  doc_types/*.yaml      # one schema per doc_type (fields, boosts, display color)
  entities/**/*.json    # entities, with related_to edges
  connectors/*.py       # optional ingestion scripts (MCP → entities)

Storage defaults to SQLite + FTS5 (zero external services). The backend sits behind a pluggable interface with two implementations: SQLite (default, local/dev) and OpenSearch (select with search.backend: opensearch — see Using the brain from a cloud agent).

Where entities live — storage: file | index

Each doc_type declares its source of truth, so curated and machine-generated data don't fight over git:

• storage: index (default) — the search DB owns these entities; they are not written to files. Right for connector-pulled, high-volume, or temporal data (hundreds of services, memories, alerts) that would otherwise churn the repo.
• storage: file — JSON files under entities/<doc_type>/ are the source of truth, git-tracked and PR-reviewable. Right for curated, human/agent-authored entities.

open-index index reconciles file-backed types from disk on each run and leaves index-backed entities (written by connectors/agents) untouched. So brain.db is durable state for index-backed types — back it up or re-ingest; it's gitignored by default.

Creating a brain, step by step

open-index init <name> scaffolds the directory below; then you author two kinds of file — doc_types (schemas) and entities (instances). Sample doc_types: infra (service, datastore, dashboard, runbook, alert), sales (lead, deal, account), lending (loan, borrower, application), or personal (goal, project, person, area, note). Three runnable examples ship in examples/: support-brain, infra-brain, and personal-brain.

1. Define a doc_type

A doc_type is a concept plus its schema — one YAML file in doc_types/:

# doc_types/service.yaml
doc_type: service
description: A deployed service.
storage: file                 # file = git source of truth · index = DB-owned (default)
display:
  label_field: name
  color: "#7c3aed"
schema:
  fields:
    - { name: name,        type: string, search: syntactic, boost: 6 }   # weighted 6× in ranking
    - { name: description, type: text,   search: semantic }
    - { name: owner,       type: string, search: syntactic }
relationships:                # the correlations this type uses — optional but recommended
  - { name: "writes to",       target_doc_type: datastore }
  - { name: "is monitored by", target_doc_type: dashboard }

• boost sets per-field search weight — a hit in a boost: 6 title outranks a boost: 1 description hit 6-to-1. Optional; defaults to 1.
• relationships declares the edge vocabulary so correlations are discoverable (shown in the UI + navigation guide) and lightly validated (right target type). Optional — entities may still use undeclared meanings.

Create one with open-index add-doc-type service (writes a stub you edit), or ask your agent.

2. Add entities

An entity is one instance. For storage: file types, write one JSON per entity under entities/<doc_type>/:

// entities/service/checkout.json
{
  "doc_type": "service",
  "id": "service:checkout",
  "name": "Checkout",
  "owner": "payments-team",
  "related_to": [
    { "target": "datastore:postgres-main",     "relationship_edge_meaning": "writes to" },
    { "target": "dashboard:checkout-latency",   "relationship_edge_meaning": "is monitored by" }
  ]
}

• id must be <doc_type>:<slug>.
• related_to is the reserved correlation field present on every entity — it defines the graph edges (target + relationship_edge_meaning). This is how you say "this ticket is about that service" without any graph database.

Then open-index index (loads file-backed entities) and open-index validate.

3. Populate at scale (four ways, one validated store)

1. Manual / agent — write JSON, or open Claude Code in the folder and let it call put_entity / create_doc_type over MCP.

2. Bulk — import a file directly, or let an agent write a batch in one call with put_entities:

   open-index import issues.csv --doc-type issue --asserted-by import:jira
   open-index import export.jsonl --dry-run        # validate first, write nothing
   

   JSON arrays, JSONL, and CSV all work. Bare slugs are qualified (checkout → product:checkout), CSV scalars are coerced, and a related_to column takes target|meaning pairs separated by ;. A bad row is reported and skipped — the rest still land. --asserted-by / --confidence attribute the whole batch once instead of per row.

3. Connectors — connectors/*.py pull from an MCP server on a schedule; run with open-index ingest <name> or open-index run (c