Amadeus

Amadeus is a real-time multimodal desktop agent that brings voice, character presence, and Agent work into one interface. Speak or type naturally, delegate tasks to specialized Providers, and stay involved through visible progress, permission requests, and controls to resume or take over.

The desktop app and Host run locally, with remote services or local inference selected through model configuration. Character packs are optional: Chat and Work remain available without them.

What Amadeus is trying to solve

Voice assistants, desktop characters, and execution agents usually live in separate windows: one chats, one performs, and another works in a terminal or browser. Once a long task starts, it is difficult to see what is happening, which permission is needed, or whether the work can recover after a failure.

Amadeus connects those experiences into one loop:

1. Talk — communicate naturally: speak or type, with interruption across generation, synthesis, and physical playback.
2. Embody — make the agent present: voice, subtitles, lip sync, expression, and scene behavior share one playback timeline.
3. Act — delegate real work: the main role delegates to registered Work Providers instead of receiving every tool directly.
4. Control — stay in charge: Projects, Drafts, Artifacts, progress, permissions, diffs, and results remain visible and recoverable.

The character communicates and narrates, specialized Providers execute, and the Host owns identity, state, permissions, persistence, and recovery.

[!IMPORTANT] This repository contains buildable, runnable source. This branch targets 0.16.1 Alpha, not a packaged desktop release. Amadeus first-party code is open-source under the GNU Affero General Public License v3.0 (AGPL-3.0). Third-party code and external assets retain their own terms.

Want to run it first? See Quick start. For an introduction, start with What Amadeus is trying to solve.

Demo highlights

Click the image to watch the full 10-minute demo.

Real-time conversation and performance	Scene-aware working state
Voice, subtitles, lip sync, and expression follow actual playback.	Background work drives character behavior, scene state, and result narration.

The video demonstrates real-time voice, character performance, desktop scenes, Browser / OpenClaw tasks, and a paper-research flow. The desktop UI, Provider integration, and asset boundaries have continued to evolve, so the video is a product slice rather than a pixel-exact installation preview.

[!NOTE] The header is a brand illustration; the demo screenshots show the prototype. Characters, scenes, voices, and other third-party material in the header or demo are not granted rights by the Amadeus code license. Public source does not include character packs, model weights, reference audio, or authoring intermediates without confirmed redistribution rights.

Current capabilities

MCP and Skills remain compatible-Provider capabilities even when they share a Host registry; Main Chat cannot invoke MCP tools directly. Remote DeepSeek is the Main Chat baseline; remote ASR/TTS remain explicit compatibility routes. A local voice failure never silently uploads data or creates a second billable request.

Repository map

electron/       Electron main, preload, React renderer, and Settings
server/         authenticated local backend, Host control plane, and AUIP
core/           Main Chat runtime and session integration
agent_host/     Provider contracts, adapters, Work identity, and capabilities
asr/            Conversation / Wake recognition backends
tts/            synthesis backends, sentence pipeline, playback, and mouth signal
render/         SpriteForge runtime adapter and PixiJS renderer
wallpaper/      Electron/Lively hosts and Win32 desktop placement
vn_player/      experimental VN Player integration
assets/         Git-owned UI assets and external runtime-asset destinations
release/        public-source selection, provenance, and deterministic archive policy
config/         startup parsing, shared declarations in config/catalog/, state I/O
llm/            model clients, transports, and character prompt composition
characters/     built-in character definitions
sdk/            AUIP integration libraries
schemas/        machine-readable protocol and package contracts
examples/       external application and integration examples
skills/         Provider-scoped authoring instructions
tools/          maintenance and acceptance entry points (see tools/README.md)
tests/          deterministic product and boundary contracts
docs/           setup guides, contracts, and dated engineering evidence
architecture/   architecture sources and generated views
scripts/        platform launch and installation helpers
local_tts_infer.py  GPT-SoVITS inference integration used by tts/

main.py is not an application entry; it prints a retirement notice. The Python entry is uv run --locked --no-sync python -m server.app --port 17777, and the desktop entry is run_electron_utf8.bat on Windows or npm run electron:dev from electron/ on macOS. Both discover .venv automatically; choose an installation profile supported on your platform.

Architecture

The dashed Memory & Persona Runtime is a roadmap extension. Its two-way connection to Main Chat represents proposed context retrieval and updates; storage, memory formation, persona updat