ORBIT

Open Retrieval-Based Inference Toolkit

A self-hosted, OpenAI-compatible AI gateway for private RAG, natural-language data access, and tool-calling agents — run it in your own environment across 37+ model providers.

What is ORBIT?

ORBIT is a single API gateway that puts production controls in front of any AI model and connects it to your private data. Point one OpenAI-compatible /v1/chat endpoint at local or cloud models, ask natural-language questions over your databases and documents, and give models controlled access to tools — all behind API keys, quotas, rate limits, and audit logs you host yourself.

Reach for ORBIT when you need to:

• 💬 Query SQL, NoSQL, vector stores, REST/GraphQL APIs, Elasticsearch, and files in plain English
• 🔀 Route one API contract across local (Ollama, llama.cpp, vLLM) and cloud models
• 🛠️ Give models scoped access to MCP tools and multi-step agent workflows
• 🔒 Ship private RAG with auth, quotas, moderation, metrics, and admin controls
• 🧪 Prototype locally, then keep the same architecture in production

Comparisons: ORBIT vs. Open WebUI · ORBIT vs. LiteLLM

⭐ If ORBIT is useful to you, a star helps more developers find it.

🏗️ Architecture

🚀 Quick Start

[!WARNING] Do not clone and deploy the main branch in production. Use the latest stable release tarball from GitHub Releases instead. The main branch is intended for development and testing and may include unreleased changes.

Latest stable release tarball (Linux/macOS):

curl -LO https://github.com/schmitech/orbit/releases/download/v2.8.3/orbit-2.8.3.tar.gz
tar -xzf orbit-2.8.3.tar.gz && cd orbit-2.8.3
./install/setup.sh        # add --wizard for interactive setup
./bin/orbit.sh start

Docker Compose from main — for local development/testing with Ollama and a lightweight local model:

git clone https://github.com/schmitech/orbit.git
cd orbit/docker
docker compose up -d
# NVIDIA GPU: docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d

Windows (native): see the Windows installation guide.

Verify the gateway:

curl -X POST http://localhost:3000/v1/chat \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: default-key' \
  -H 'X-Session-ID: local-test' \
  -d '{"messages": [{"role": "user", "content": "Summarize ORBIT in one sentence."}], "stream": false}'

Then open the admin dashboard at http://localhost:3000/admin (admin / admin123) for metrics, latency, sessions, adapters, logs, and health.

📖 Full setup: Docker Guide · Tutorial

✨ Features

📚 Deep dive: Docs index · Adapter guide · Configuration · Cookbook

💡 Use Cases

Same gateway, different jobs. Each use case below maps to a recurring pain that teams across industries are trying to solve with AI — with a short demo as proof.