Aithy

      ..:::::..
   .:+#########+:.
  :###=:....:=###:
 .##+  AITHY   +##.
 :##  local AI  ##:
 .##+  //////  +##.
  :###=::::=###:
   .:+#####:+.
      ':::'

Aithy is a private local AI runtime: easy to run, inspectable in the browser, and ready for local models, cloud providers, eligible Grok subscriptions, and Mesh-shared home GPUs.

It is pronounced ay-thee. It is named after my cat.

Aithy is useful as a personal agent today, but it is also built for the messy place where agent research becomes real software: durable sessions, sandboxed tools, local inference, retrieval, memory, skills, background work, and multiple supervised services running together in an inspectable local product.

Why Aithy

• Run it locally. Start Aithy, open the browser control surface, and use an inspectable agent workspace without wiring an agent framework by hand.
• Use the models you already have. Choose managed local llama.cpp models, normal cloud/API providers, Ax named profiles including OpenAI Responses and OpenAI-compatible servers, custom deployment endpoints, or an eligible Grok subscription.
• Make your Grok subscription useful. Grok subscription sign-in can power chat and web.search without pasting an API key into Aithy. For people who already pay for Grok, that can avoid setting up separate per-token API-key billing for supported usage, subject to xAI eligibility and limits.
• Share a stronger machine. Pair Aithy on your laptop with Aithy on a GPU box over LAN Mesh, then use validated family inference/search services without copying secrets around.
• Keep the agent visible. Sessions, memory, skills, attentions, artifacts, usage, permissions, training-data capture, sandbox state, and runtime services are part of the product, not hidden logs. Chat streams assistant responses, shows model-reported progress, and keeps inline artifacts and explicit approval bubbles visible while work is happening.

Quick Start

Download a build

Download the latest packaged build from GitHub Releases, choose the archive for your OS, unpack it, and run Aithy:

gh release download --repo dosco/aithy --pattern 'aithy-*-darwin-arm64.tar.gz'
tar -xzf aithy-*-darwin-arm64.tar.gz
cd aithy-*-darwin-arm64
./aithy

Open http://127.0.0.1:3000. Use --host and --port if you want a different bind address.

Release archives are named by platform:

• aithy-v*-darwin-arm64.tar.gz for Apple Silicon Macs.
• aithy-v*-linux-x64-gnu.tar.gz for Linux x64.
• aithy-v*-linux-arm64-gnu.tar.gz for Linux ARM64.

Intel Mac and Windows archives are not published yet because the packaged sandbox runtime is not available for those targets.

Start from code

Use this path if you want to run from the repo, hack on Aithy, or follow the code as it changes. You need Bun 1.4.0 or newer.

bun install
bun run start

The welcome screen walks through your profile, provider, model, key or sign-in flow, verified thinking level and service tier choices, sandbox settings, and local inference options.

Local state lives at ~/.config/aithy/default/ by default. Use the web UI and persisted settings for provider-scoped model profiles, search profiles, secrets, sandbox settings, Mesh, and local inference.

Every Aithy is a full Aithy. There is no separate inference-only daemon: to share a GPU box with laptops on the same LAN, run Aithy on each machine, open Mesh, pair them, and mark the relationship level you actually trust.

Packaged Releases

Pushing a v* tag runs the GitHub Actions release flow. It checks the repo, builds the production web app, packages supported portable archives, smoke-tests the Linux x64 package, and attaches archives plus SHA-256 checksums to the GitHub release.

Each archive contains one pinned Bun runtime, bundled app JavaScript, built client assets, a target-specific native vendor slice, and external worker bundles only for sandbox and local inference. The packaged aithy command runs the web runtime, queue service, and agent/background queues inside one coordinator process while keeping sandbox-worker and local-inference-worker as supervised child processes.

Screenshots

These light-mode screenshots are captured from the real app with a disposable local profile, demo-safe runtime and usage records, and no real credentials.

Use the Models You Already Have

Aithy lets the model source be a runtime choice instead of a project rewrite.

Cloud setup uses Ax's reviewed named profiles for OpenAI Chat, OpenAI Responses, OpenRouter, Together, Groq, Azure OpenAI, and compatible local or hosted endpoints. Profiles supply their authentication and endpoint requirements, so optional API keys remain optional and deployments such as Azure expose their required resource and deployment fields. Non-secret endpoint values are persisted with the provider profile; credentials remain in Bun secrets.

When Ax publishes verified capability metadata for the selected model, Inference shows portable thinking levels and explicit service tiers. Thinking levels can map to the same provider-native setting. Service tier Auto delegates policy to the provider, while Standard, Flex, and Priority appear only when verified for that provider and model. Dynamic profiles with no bundled model list use their provider-level capabilities; unknown models on static providers do not receive unverified advanced options.

flowchart LR
  User["You"] --> UI["Aithy browser control surface"]
  UI --> Source{"Model source"}
  Source --> Local["This Aithy: managed llama.cpp"]
  Source --> Grok["This Aithy: eligible Grok subscription"]
  Source --> Cloud["This Aithy: cloud/API provider"]
  Source --> Family["Family Aithy over Mesh"]
  Local --> Agent["Agent pipeline"]
  Grok --> Agent
  Cloud --> Agent
  Family --> Agent

Local inference uses a managed llama.cpp llama-server release pinned in package.json, with Qwen GGUF chat models plus local Qwen embedding and reranker models. A saved path to your own compatible llama-server can be used instead.

If the Runtime Console shows local-inference-worker as DEGRADED, the local router for local chat, memory embeddings, or reranking is unavailable or recovering. Cloud-provider chat can still work when Local is not selected, and the console shows the current error plus any automatic retry countdown.

The Grok subscription provider uses OAuth/PKCE sign-in. Tokens stay in local Aithy secrets, and the connected provider can be used for chat and web.search when the subscription is eligible for the underlying xAI access.

Share Inference With Mesh

Aithy Mesh is for the home-lab shape a lot of local AI users already have: a laptop where you work, and a stronger machine somewhere nearby.

flowchart LR
  Laptop["Laptop Aithy"] --> Pair["Explicit pairing"]
  Gpu["GPU-box Aithy"] --> Pair
  Pair --> Trust["Mutual family trust"]
  Trust --> Catalog["Live validated service catalog"]
  Laptop --> Proxy["Local OpenAI-compatible proxy"]
  Proxy --> Rpc["Pinned TLS / HTTP3 mesh RPC"]
  Rpc --> Gpu
  Gpu --> Services["Local, cloud, Grok, search services"]

Mesh discovery is LAN-only. It advertises identity and connection metadata, not model lists, provider URLs, API keys, prompts, sessions, memories, or service catalogs. Family catalogs are live RPC calls, only mutual family peers can return them, and shared providers are never recursively re-shared.

Built for Real Agent Work

Aithy is not just a chat box with a pile of tools. It treats the agent as a local runtime system.

Assistant responses stream into the active chat as they are generated. When the agent needs structured input, the latest clarification can render choices, a date field, or a number field; the selected labels are sent as an ordinary next turn and remain visible in the durab