Aithy

      ..:::::..
   .:+#########+:.
  :###=:....:=###:
 .##+  AITHY   +##.
 :##  local AI  ##:
 .##+  //////  +##.
  :###=::::=###:
   .:+#####:+.
      ':::'

Aithy is a private local AI runtime: easy to run, inspectable in the browser, and ready for local models, cloud providers, eligible Grok subscriptions, and Mesh-shared home GPUs.

It is pronounced ay-thee. It is named after my cat.

Aithy is useful as a personal agent today, but it is also built for the messy place where agent research becomes real software: durable sessions, sandboxed tools, local inference, retrieval, memory, skills, background work, and multiple supervised services running together in an inspectable local product.

Why Aithy

• Run it locally. Start Aithy, open the browser control surface, and use an inspectable agent workspace without wiring an agent framework by hand.
• Use the models you already have. Choose managed local llama.cpp models, normal cloud/API providers, custom OpenAI-compatible endpoints, or an eligible Grok subscription.
• Make your Grok subscription useful. Grok subscription sign-in can power chat and web.search without pasting an API key into Aithy. For people who already pay for Grok, that can avoid setting up separate per-token API-key billing for supported usage, subject to xAI eligibility and limits.
• Share a stronger machine. Pair Aithy on your laptop with Aithy on a GPU box over LAN Mesh, then use validated family inference/search services without copying secrets around.
• Keep the agent visible. Sessions, memory, skills, attentions, artifacts, usage, permissions, training-data capture, sandbox state, and runtime services are part of the product, not hidden logs. Chat streams assistant responses, shows model-reported progress, and keeps inline artifacts and explicit approval bubbles visible while work is happening.

Quick Start

Download a build

Download the latest packaged build from GitHub Releases, choose the archive for your OS, unpack it, and run Aithy:

gh release download --repo dosco/aithy --pattern 'aithy-*-darwin-arm64.tar.gz'
tar -xzf aithy-*-darwin-arm64.tar.gz
cd aithy-*-darwin-arm64
./aithy

Open http://127.0.0.1:3000. Use --host and --port if you want a different bind address.

Release archives are named by platform:

• aithy-v*-darwin-arm64.tar.gz for Apple Silicon Macs.
• aithy-v*-linux-x64-gnu.tar.gz for Linux x64.
• aithy-v*-linux-arm64-gnu.tar.gz for Linux ARM64.

Intel Mac and Windows archives are not published yet because the packaged sandbox runtime is not available for those targets.

Start from code

Use this path if you want to run from the repo, hack on Aithy, or follow the code as it changes. You need Bun 1.3.14 or newer.

bun install
bun run start

The welcome screen walks through your profile, provider, model, key or sign-in flow, sandbox settings, and local inference options.

Local state lives at ~/.config/aithy/default/ by default. Use the web UI and persisted settings for provider-scoped model profiles, search profiles, secrets, sandbox settings, Mesh, and local inference.

Every Aithy is a full Aithy. There is no separate inference-only daemon: to share a GPU box with laptops on the same LAN, run Aithy on each machine, open Mesh, pair them, and mark the relationship level you actually trust.

Packaged Releases

Pushing a v* tag runs the GitHub Actions release flow. It checks the repo, builds the production web app, packages supported portable archives, smoke-tests the Linux x64 package, and attaches archives plus SHA-256 checksums to the GitHub release.

Each archive contains one pinned Bun runtime, bundled app JavaScript, built client assets, a target-specific native vendor slice, and external worker bundles only for sandbox and local inference. The packaged aithy command runs the web runtime, queue service, and agent/background queues inside one coordinator process while keeping sandbox-worker and local-inference-worker as supervised child processes.

Screenshots

These light-mode screenshots are captured from the real app with a disposable local profile, demo-safe runtime and usage records, and no real credentials.

Use the Models You Already Have

Aithy lets the model source be a runtime choice instead of a project rewrite.

flowchart LR
  User["You"] --> UI["Aithy browser control surface"]
  UI --> Source{"Model source"}
  Source --> Local["This Aithy: managed llama.cpp"]
  Source --> Grok["This Aithy: eligible Grok subscription"]
  Source --> Cloud["This Aithy: cloud/API provider"]
  Source --> Family["Family Aithy over Mesh"]
  Local --> Agent["Agent pipeline"]
  Grok --> Agent
  Cloud --> Agent
  Family --> Agent

Local inference uses a managed llama.cpp llama-server release pinned in package.json, with Qwen GGUF chat models plus local Qwen embedding and reranker models. A saved path to your own compatible llama-server can be used instead.

If the Runtime Console shows local-inference-worker as DEGRADED, the local router for local chat, memory embeddings, or reranking is unavailable or recovering. Cloud-provider chat can still work when Local is not selected, and the console shows the current error plus any automatic retry countdown.

The Grok subscription provider uses OAuth/PKCE sign-in. Tokens stay in local Aithy secrets, and the connected provider can be used for chat and web.search when the subscription is eligible for the underlying xAI access.

Share Inference With Mesh

Aithy Mesh is for the home-lab shape a lot of local AI users already have: a laptop where you work, and a stronger machine somewhere nearby.

flowchart LR
  Laptop["Laptop Aithy"] --> Pair["Explicit pairing"]
  Gpu["GPU-box Aithy"] --> Pair
  Pair --> Trust["Mutual family trust"]
  Trust --> Catalog["Live validated service catalog"]
  Laptop --> Proxy["Local OpenAI-compatible proxy"]
  Proxy --> Rpc["Pinned TLS / HTTP3 mesh RPC"]
  Rpc --> Gpu
  Gpu --> Services["Local, cloud, Grok, search services"]

Mesh discovery is LAN-only. It advertises identity and connection metadata, not model lists, provider URLs, API keys, prompts, sessions, memories, or service catalogs. Family catalogs are live RPC calls, only mutual family peers can return them, and shared providers are never recursively re-shared.

Built for Real Agent Work

Aithy is not just a chat box with a pile of tools. It treats the agent as a local runtime system.

Assistant responses stream into the active chat as they are generated. When the agent needs structured input, the latest clarification can render choices, a date field, or a number field; the selected labels are sent as an ordinary next turn and remain visible in the durable transcript.

flowchart TD
  Browser["Browser UI"] --> Web["web runtime"]
  Web --> Queue["queue-service"]
  Queue --> Agent["agent-worker"]
  Queue --> Sandbox["sandbox-worker"]
  Queue --> Local["local-inference-worker"]
  Agent --> SQLite["SQLite state"]
  SQLite --> Knowledge["Knowledge bundles, concepts, links, and proposals"]
  Sandbox --> Workspace["sandbox workspace"]
  Local --> Llama["llama.cpp router"]
  Local --> SQLite
  Agent --> Artifacts["artifacts and outbox"]

The runtime services are web, queue-service, agent-worker, sandbox-worker, and local-inference-worker. In source/dev mode they run as separate supervised processes. In packaged releases, web, queue-service, and agent-worker share the coordinator process, while sandbox and local inference stay outside that process for native dependency and safety boundaries.

Under the hood, Aithy uses Ax and Ax Agent with an RLM-style, DSPy-inspired flow: context distillation, JavaScript runtime execution, and final response generation. Deterministic work such as parsing, filterin