Home Generative Agent
Talk to your home.
[![GitHub Release][releases-shield]][releases] [![HACS][hacs-shield]][hacs] [![GitHub Stars][stars-shield]][stars] [![GitHub Activity][commits-shield]][commits] ![License][license-shield]
![Project Maintenance][maintenance-shield]
A Home Assistant integration that brings a generative AI agent into your smart home. Talk to your home, create automations in plain English, analyze camera footage, and get proactive alerts — all powered by your choice of cloud or local LLMs. HGA is a single integration that gives you conversational control over every HA entity, camera understanding with face recognition, long-term semantic memory, and the Sentinel anomaly engine.
Creating an automation in plain English — the agent writes the YAML, registers it, and it shows up in the HA automation editor.
Why HGA?
Most AI conversation integrations are prompt passthroughs: they forward your words to an LLM and read back the answer. HGA is a full agent built on LangGraph — it uses tools to control entities, query history, watch cameras, and write real HA automations; it keeps long-term semantic memory in pgvector so it remembers your preferences across conversations; and its Sentinel anomaly engine keeps safety decisions deterministic, with the LLM advising but never actuating. Everything runs against the model provider you choose — including fully local, so no data has to leave your home.
Features
| Feature | What it does |
|---|---|
| Conversational control | Talk to your home in natural language. Turn things on, check status, ask questions. |
| Automation creation | Describe what you want in chat and the agent writes and registers the HA automation. With the Critical Action PIN enabled, an automation that would unlock a door or open a garage is held for PIN confirmation before it is installed — same as the direct command. |
| Camera & image analysis | Ask the agent what it sees in any camera. Proactive motion-triggered analysis with anomaly detection. Works with Axis, Ring via ring-mqtt, Reolink, UniFi Protect, and any camera that exposes a motion entity or recording state in HA — see Camera Entities for setup notes (battery Ring cameras need a snapshot-mode tweak). |
| Sentinel anomaly detection | Deterministic rules watch for security and safety issues (unlocked locks, open entries, unknown people — the unknown-person rules require the face-service) and alert your phone. Optional LLM-powered triage and rule discovery — covering power, battery, and environmental sensors (temperature, humidity, CO₂, air quality, …). Approved discovery rules can be inspected, deactivated, reactivated, and surgically repaired via HA services. |
| Face recognition | Identify people in camera frames and personalize alerts. |
| Long-term memory | Semantic search over past conversations. The agent remembers your preferences and context. |
| Streaming responses | First tokens appear word-by-word in the HA conversation UI — no waiting for the full response. |
| Built-in speech-to-text | STT provider for Assist pipelines, backed by the OpenAI Whisper API or a fully local OpenAI-compatible server — e.g. Speaches running faster-whisper on any machine Home Assistant can reach; it does not have to be the box that serves your LLM — see STT setup. |
| Built-in text-to-speech | TTS provider for Assist pipelines, backed by the OpenAI speech API or a local OpenAI-compatible server (one Speaches container can serve both STT and TTS, with Kokoro or piper voices), so the whole voice pipeline can stay on your network — see TTS setup. |
| Cloud and edge models | Use OpenAI, Gemini, Anthropic, or run everything locally with Ollama or any OpenAI-compatible server. |
Screenshots
Camera analysis
Long-term memory with semantic search
Proactive camera notifications
Real-time camera alert mobile device notifications
Anomaly detection notification
Requirements
| Requirement | Notes |
|---|---|
| Home Assistant | 2025.5.0 minimum; 2026.4.0+ for streaming responses; 2026.9 (probatio schema core) supported |
| HACS | Required for the recommended install path; manual install is also supported |
| PostgreSQL with pgvector | Provided as a bundled HA app (step 1 below) |
| Model provider | At least one of: OpenAI, Gemini, Anthropic, Ollama, or any OpenAI-compatible server |
| Edge GPU server (optional) | Ollama, vLLM, llama.cpp, or LiteLLM for local model serving |
| face-service (optional) | An external service for face recognition in camera analysis. Also powers Sentinel's unknown-person rules, which never fire unless face recognition is enabled (the face_recognition option, off by default) with this service configured |
| Speech-to-text (optional) | An OpenAI API key (Whisper API), or a local server exposing the OpenAI /v1/audio/transcriptions endpoint, such as Speaches serving faster-whisper. The server can run on any machine Home Assistant can reach — it need not be the LLM box. A GPU with about 2 GB of VRAM is recommended for faster-whisper-large-v3-turbo; smaller whisper models run on CPU. See STT setup |
| Text-to-speech (optional) | An OpenAI API key (speech API), or a local server exposing the OpenAI /v1/audio/speech endpoint — the same Speaches container can serve both. piper voices synthesize on any CPU; Kokoro needs a modern CPU (AVX2) or a recent GPU. Audio conversion for voice satellites uses the ffmpeg bundled with Home Assistant. See TTS setup |
Quick Start
Get the basic conversational agent running in seven steps. See the full installation guide for optional apps (edge models, face recognition, a local speech server).
1. Install the PostgreSQL with pgvector app.
Requires Home Assistant OS or Supervised (apps are not available on HA Container or Core).
Click the button below to add the repository, then install and configure the app per its documentation.
If the button doesn't work, add the repository manually: Settings → Apps → App Store → ⋮ → Repositories, enter
https://github.com/goruck/addon-postgres-pgvector, then search for and installpostgres_pgvector.
2. Install Home Generative Agent from HACS.
3. Restart Home Assistant.
4. Add the integration: Settings → Devices & Services → Add Integration → search Home Generative Agent → complete the initial instruction screen.
5. Add a Model Provider: on the integration page click + Model Provider. The first step asks Edge or Cloud — that choice decides which provider types the next step offers (Edge → Ollama, OpenAI Compatible; Cloud → OpenAI, Gemini, Anthropic). A provider must exist before you can run Setup. Click + Model Provider again for each additional provider you want — reconfiguring an existing one replaces it in place rather than adding another.
6. Open the integration page and click + Setup. Choose a setup mode:
- Basic — enables all features with recommended defaults and creates the database subentry automatically. No database prompt appears.
- Advanced — configure each feature individually; includes a database configuration step.
7. Set as your voice assistant: Settings → Voice Assistants → select Home Generative Agent as the conversation agent.
You can now open the HA Assist panel and start talking to your home.
Optional — voice in and out: click + STT Provider and + TTS Provider on the integration page to add HGA's own speech-to-text and text-to-speech engines (OpenAI, or a local Speaches server on any machine), then select them in your Assist pipeline. See STT setup and TTS setup.
Documentation
| Guide | Contents |
|---|---|
| Installation | HACS install, manual install, optional apps (Ollama, face recognition, local speech server for STT/TTS) |
| Configuration | Model providers, features, per-model thinking/reasoning & budget, Tool Retrieval (RAG), per-tool exclusions & always-included tools, LLM API, STT and TTS (OpenAI or local), YAML mode, Critical Action PIN, camera description language & extra VLM instructions, UI languages (en/cs/ru/tr) |
| Sentinel | Anomaly detection pipeline, built-in rules, triage, baseline, blueprints, notification quiet hours, services API, health sensor |
| [Camera Entities](docs/camera |