orbit

For GPU acceleration:

docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d

Adapter wiring and sample domains live in config/adapters/ and examples/intent-templates/. See the full Docker Guide for GPU mode, volumes, and configuration.

Demo

Why ORBIT?

| Common problem | What ORBIT provides | | :--- | :--- | | One SDK per provider, with rewrites when you switch | One OpenAI-compatible API across local and hosted providers | | Separate systems for inference, retrieval, tools, and chat history | One gateway for model calls, adapters, tools, conversations, and clients | | RAG limited to vector search over clean documents | Retrieval over SQL, NoSQL, HTTP, GraphQL, files, web content, and vector stores | | Glue scripts between prompts and business systems | Intent adapters, composite adapters, diagnostics, and reusable templates | | Privacy-sensitive data sent through third-party services by default | Self-hosted deployment with local models, local embeddings, API keys, RBAC, audit logs, and rate limits | | Provider failures cascading into application failures | Circuit breakers, failover, parallel fan-out, and quota-aware throttling |

Core Capabilities

AI Gateway

• Route requests across local and hosted models.
• Use one OpenAI-compatible chat API with existing clients and SDKs.
• Stream responses with failover, moderation hooks, and rate limiting.
• Connect through the web chat, Node SDK, or OpenAI-style clients.

Retrieval And Adapters

• Retrieve context from databases, APIs, files, web content, and vector stores.
• Use intent-based retrieval for natural-language questions over structured data.
• Fan one prompt across multiple sources with composite adapters.
• Build and debug data-backed assistants with template diagnostics and autocomplete.

Private RAG And Conversations

• Run private RAG over PDFs, documents, images, manuals, contracts, and knowledge bases.
• Reuse retrieved context through conversation threading and cached datasets.
• Handle multilingual conversations across 100+ languages.
• Keep deployments self-hosted for privacy-sensitive environments.

Media Generation

• Generate images from text prompts using DALL-E, Stability AI, and other providers.
• Generate videos from text prompts using Google Veo 2 — prompts are automatically enriched with motion, camera movement, and lighting detail before generation.
• Generated media is persisted server-side and delivered via stable URLs, keeping large binary payloads off the wire.
• Image and video generation follow the same adapter type system as every other capability — no special-casing required.

Tools, Agents, And Production Controls

• Expose controlled tools through MCP for agent clients.
• Invoke specialized adapters during a conversation with cross-adapter skills.
• Operate with API keys, RBAC, quotas, audit logs, rate limits, and circuit breakers.
• Add voice assistant support through audio adapters.

Who Is ORBIT For?

• Developers building internal AI apps that need real company data, not isolated chat.
• Teams that need private RAG over documents, databases, APIs, and operational systems.
• Companies avoiding long-term lock-in to a single LLM provider or hosted AI platform.
• Engineers connecting AI to SQL, NoSQL, REST, GraphQL, files, and document stores.
• Builders who want OpenAI-compatible APIs with self-hosted control.

ORBIT is probably more than you need if you only want a thin wrapper around one LLM provider.

What Makes ORBIT Different?

ORBIT is not only a model router. It handles the layers that usually become custom infrastructure in production RAG systems: retrieval, tools, adapters, conversations, access control, and operational safeguards.

• Retrieval beyond vector search: use intent templates and adapters for structured databases, APIs, files, web content, and vector stores. Intent SQL RAG
• Data source support: query SQL, MongoDB, Elasticsearch, REST, GraphQL, DuckDB, files, and composite sources through one gateway. Composite adapters
• Local and hosted models: run private workloads on Ollama, llama.cpp, vLLM, or other local providers, while still supporting hosted LLMs where appropriate.
• Production controls included: use API keys, RBAC, quotas, audit logging, moderation, rate limits, and circuit breakers. Rate limiting
• Agent-ready protocol support: expose ORBIT-backed chat, RAG, and adapter tools through MCP. MCP / OpenClaw walkthrough

Example Use Cases

| Use case | Start here | | :--- | :--- | | Chat with a local model through an OpenAI-compatible API | Step-by-step tutorial | | Ask Postgres, MySQL, MongoDB, DuckDB, or Elasticsearch questions in natural language | Database copilot | | Query SQL + NoSQL + REST APIs in one prompt | Composite adapters | | Upload files and get grounded answers | File-upload RAG | | Deploy a private AI gateway for regulated data | Private gateway cookbook | | Run ORBIT as an MCP tool server for agents | [MCP / OpenClaw walkth