Voice-First Desktop AI Assistant

Build Faster. Automate Workflows. Control your Desktop with Voice Commands.

Speak your command. IRIS executes it.

A voice-first neural execution system powered by Gemini 3.1 Live API with real-time WebRTC audio, biometric security, and autonomous system control.

📑 Table of Contents

• ⚡ Overview
• 🎯 What is Voice-First?
• ✨ Core Features
• 🔐 Code Protection & Security
• 💰 Sponsorship Tiers
• 🏗️ Architecture
• 💻 Tech Stack
• 🚀 Installation & Setup
• 📁 Project Structure
• 🧠 Development Philosophy
• 🤝 Contributing
• 🧩 Extending IRIS
• 🧠 Roadmap
• ⚠️ Disclaimer
• 👨‍💻 Architect
• 📜 License

⚡ Overview

IRIS is not a chatbot.

It is a Voice-First Desktop AI Assistant that executes real-world actions across your system, applications, and devices—powered by Gemini 3.1 Live API with real-time bidirectional audio processing.

Speak naturally. IRIS understands intent. Real actions execute instantly.

What Makes IRIS Different?

✅ Voice-First Design – Optimized for natural speech input with real-time WebRTC audio streaming
✅ Proprietary Agent Logic – Heavily protected, production-grade agentic orchestration
✅ Production-Ready Security – V8 bytecode + ASAR integrity validation + window isolation
✅ No Code Exposure – Core agent and tools are completely hidden from public source
✅ Autonomous Execution – LangGraph-powered state machine with dynamic tool orchestration

✨ What's New In v1.7.0

• Added interactive Quiz and Tic Tac Toe widgets.
• Deeper System Control for IRIS.
• Completely revamped Dashboard UI and Settings UI for a seamless experience.
• Updated AI State colors: Thinking (Blue), Speaking (Green), and Standby (White).
• Enhanced Infrastructure: New server added for more stability and control.
• Important: All previous versions prior to 1.7.0 are completely deprecated and no longer function.
• Free Tier Rate Limit: Free tier is now limited to 12 turns and 5 tool calls per day (resets 12:00 PM - 2:00 PM).
• Bug fixes for Email, WhatsApp, Image Generation, Ghost Control, Wallpaper Engine, Camera, and Screen Context for AI.

🪡 Open Core Model

IRIS follows an Open Core development model.

The public repository includes:

• Desktop application framework
• User interface
• Core infrastructure
• Selected integrations
• Community-facing examples

The following production components are private:

• Core voice orchestration engine
• Advanced tool execution logic
• Internal automation systems
• Production-grade implementations
• Certain premium modules

GitHub Sponsors receive access to additional documentation, implementation examples, architecture breakdowns, and development resources depending on tier.

Sponsorship does not include access to the complete private source code.

🎯 What is Voice-First?

Traditional AI assistants are text-first: you type → they respond → you read.

IRIS is voice-first: you speak → they listen & execute → actions happen in real-time.

Real-Time Audio Processing

Your Voice
    ↓ (WebRTC Stream)
Gemini 3.1 Live API (Real-time)
    ↓ (Intent Recognition)
LangGraph Agent Orchestration
    ↓ (Tool Selection)
Protected Tool Execution
    ↓ (System Actions)
Results Streamed Back to You

• Latency: < 500ms end-to-end (including network)
• Quality: Full duplex (talk while agent responds)
• Models: Gemini 3.1 Live API (primary) + Groq (Fast Responses) + Hugging Face (Open-Sourced + Local Models)
• Search: Tavily for real-time web data

No local-only limitations. IRIS connects to cloud AI, search engines, and APIs for maximum intelligence.

✨ Core Features & System Capabilities

✨ Special Features

Autonomous voice activation hooks, advanced screen character peeling, and phantom inline input overlays.

• Wake Up Word Activation: IRIS is configured for hands-free local startup. Speaking the wake word automatically opens the assistant window, performs local telemetry diagnostics, and checks real-time atmospheric updates.
  • Commands: "Hey, IRIS", "IRIS", "Wake up, IRIS"
• Phantom Control (Ghost Keyboard): Inline typing injection overlay. Activating the shortcut creates a phantom input hook to inject typed keystrokes anywhere on the OS, integrating cleanly with VS Code.
  • Commands: "Press Ctrl + Alt + Space", "Activate Phantom Typer", "Start Ghost Typer"
• ScreenPeeler (Multimodal AI OCR): Intelligent rectangular region screen selection. Takes a high-resolution snapshot of any screen coordinate area, runs local/cloud multimodal extraction, and populates extracted text to your clipboard.
  • Commands: "Press Ctrl + Alt + X", "Extract text from active workspace", "Scan system screen portion"
• Small Ghost Overlay (Ctrl + Shift + I): [PRO Exclusive] Launches a sleek, floating quick-access mini overlay at your cursor for immediate keyboard/voice commands without opening the main OS dashboard.
  • Commands: "Press Ctrl + Shift + I", "Open quick overlay", "Toggle ghost mini panel"

📂 System & File Management

Complete native file system and directory access with app process lifecycle controls.

• Open App: Native application lifecycle initialization.
  • Commands: "Open Spotify", "Launch VS Code", "Start Google Chrome"
• Close App: Instant process termination hook.
  • Commands: "Close Photoshop", "Kill the Chrome process", "Stop Node"
• Create Folder: Directory structure generator.
  • Commands: "Create a folder named assets in my current directory", "Make folder UI under components"
• Read & Write Files: Disk file writing and code extraction.
  • Commands: "Read the index.js file inside the root", "Write a server.js file with simple express setup"
• Smart Drop Zones: Autonomous sorting algorithms for system files.
  • Commands: "Sort my downloads folder", "Organize my chaotic project directories"

🧠 Vector Search & Local Knowledge

Semantic ingestion using local Vector databases and direct multimodal vision APIs.

• Index Folder: Index folder contents into a local semantic database.
  • Commands: "Index my src folder", "Embed my docs folder for search"
• Smart File Search: Vector-based local file retrieval.
  • Commands: "Find files related to user authentication", "Search for codebase configuration hooks"
• Analyze Photo & Gallery: OCR and direct multimodal layout processing.
  • Commands: "Scan my screenshot folder", "Analyze this error screenshot and find a solution"

💻 Developer & Terminal Tools

Globally accessible NPM package with tunneling and secure CLI execution.

• Run Terminal: Native shell script/CLI executor.
  • Commands: "Run npm run build", "Execute git status", "Run typescript checker"
• Deploy Wormhole: Localhost tunnels exposing local servers to the public internet.
  • Commands: "Expose port 3000 to the public internet", "Open local server to external connection"
• Execute Sequence / Macro: JSON-based workflow sequence triggering.
  • Commands: "Run the development startup sequence", "Execute my custom deploy macro"
• Manage PC Settings: Control OS-level settings like Wi-Fi, Bluetooth, Audio, and Display.
  • Commands: "Turn off Wi-Fi on my PC", "Open the sound settings"

🎯 Desktop UI & Automation

AI-driven coordinate cursor control, scroll tracking, and screen peeler OCR.

• Teleport Windows: Desktop window movement, resizing, and alignment.
  • Commands: "Move this active window to the left side", "Minimize active window", "Maximize terminal"
• Click & Scroll on Screen: Cursor control with AI coordinate calculation.
  • Commands: "Click the login button", "Scroll down fifty percent", "Click at coordinates 800 by 600"
• Screen Peeler & Phantom Typer: Instant OCR extraction to code editor.
  • Commands: "Extract code from active window", "Type my secure email address in the active input box"
• Custom Protocols: Build and execute multi-step automated workflows natively in the OS.
  • Commands: "Activate Coding Mode", "Start Hardcore Gaming Protocol"
• AI Wallpaper Engine: Dynamically change PC wallpaper with natural language queries.
  • Commands: "Change my wallpaper to a cyberpunk city", "Set background to a mountain sunset"

💾 Memory & Information

Persistent identity tracking, note management, and remote inbox integrations.

• Core Memory Ingestion: Saves details into permanent memory database.