aipointer

If AIPointer ⦿ is useful to you, a star 🌟 on GitHub helps the project stay alive.

What it is

AIPointer is an open-source desktop overlay. You hold a key (default: Right-Cmd on macOS, Right-Ctrl on Windows/Linux), a glassmorphism box pops up next to your cursor, you ask a question, and a vision-capable LLM answers about whatever's around the cursor. A screenshot of the cursor region, your prompt, and (optionally) your clipboard get sent to the provider you configured. You keep your own API key, you pay for your own tokens, nothing is logged anywhere.

BSL-1.1 source-available, no framework lock-in, no cloud account. For longer autonomous tasks, AIPointer points you at Skales, the same author's larger AI agent.

Common use cases

AIPointer is useful when you want to ask an AI about something on your screen without copying, pasting, or switching apps:

• Quick translation of text you're reading in any app or website
• Explain code in your editor without leaving it
• Identify an object, product, landmark, or chart from a screenshot
• Summarize a long article or document you have open
• Ask about your files — select up to 5 files in Finder / Explorer, press the AIPointer hotkey, and they're attached to your next query automatically. Zero clicks. Hit Enter without typing and AIPointer auto-summarizes them.
• Get a reply suggestion for a message visible on screen
• Define a word without leaving your current app
• Solve a math or logic problem by pointing at it
• Voice queries when typing isn't convenient

It works as an alternative to switching to ChatGPT, Claude, or Gemini in a browser tab — you stay where you are, the answer comes to you.

Demo

Why use it

• Cursor-anchored. It answers about what you're already looking at, not what you have to describe in text.
• Fast. Vision-capable Gemini 3 Flash by default. Sub-2-second answers for most questions.
• Multi-provider, with a fallback chain. Bring your own OpenRouter key (recommended) or a direct Anthropic, OpenAI, or Google Gemini key. Pick a primary provider and stack up to 3 fallbacks under it, each pinning its own model; the router walks the chain on transient failures (5xx / 429). A wrong or expired key still stops on that provider so you can fix it.
• Pin any model you like. Each provider row has a ↻ Refresh models button that pulls the live model list with your saved key. Or leave it on Auto and let AIPointer pick the cheapest viable one.
• Attach files without leaving your file manager. Select up to 5 files in Finder or Explorer, press the hotkey, and they're on your next query — zero clicks, no drag-and-drop. Or use the paperclip (/attach). Images go in as vision, text / JSON / CSV / Markdown / code is inlined, PDFs and Office docs are referenced by name. Press Enter with files queued and no prompt, and it summarizes them.
• The whole voice loop can run on your machine. Read-aloud via Kokoro and speech-to-text via Whisper both run on-device — multilingual, no API key, no cloud round-trip. Or stay on your system voice (the default, zero setup) or a cloud provider. See the table below.
• Works across multiple displays. The box, the screenshot, the drag-to-capture region and your typing all happen on whichever screen your cursor is on.
• Agentic when it helps. Seven built-in tools: fetch a URL, open a URL, copy text, save the answer as a styled document, reveal your workspace, read clipboard, and launch a desktop app from a curated whitelist. Every action tool sits behind a green-check / red-x approval.
• Child Mode. A kid-friendly response layer behind a PIN: per-language safe-browsing whitelist (EN + DE), stricter HTML rendering, a restricted tool set, voice-first with slower TTS. Visually identical to Adult mode — it is a behavior layer, not a different app.
• Voice commands in 7 languages. "open settings" / "einstellungen öffnen", "play <song>" / "spiele Bach", "search for X" / "such nach X", "louder / quieter / mute / volume 30". EN, DE, FR, ES, IT, PT, NL, matched before the LLM with tolerance for transcription wobble. Say "stop" / "halt" to interrupt.
• Region screenshots. While the box is open, hold the trigger again and drag a rectangle to capture exactly what you want.
• Chat-only mode. Settings → Behaviour. Opens AIPointer as a pure chat window with no screenshot attached; a camera button adds one per query when you want it.
• Templates that work. /summary, /brief, /translate, /explain, /code, /improve, /define, /solve, /reply, /identify. Plain prose works too — AIPointer detects intent across languages.
• Save the answer. A4-styled HTML, PDF, or Markdown. Print-ready.
• Yours to look at. Light and dark theme independent of your OS, three cursor accents, hotkey and mouse-wiggle activation each toggleable.
• Local-first. No telemetry, no analytics, no crash reporting. API keys live in your OS keychain. Auto-updater checks quietly once a day and can be turned off.

Version-by-version detail lives in CHANGELOG.md.

Install

Download a signed build for your OS at aipointer.app, or build from source:

git clone https://github.com/gonemedia/aipointer
cd aipointer
npm install
npm run dev

On first launch you'll be prompted to grant Accessibility (for the global key listener) and Screen Recording (for the cursor-region screenshot). Both are required on macOS. The Settings panel opens automatically the first time so you can paste an API key. Get one at:

• openrouter.ai/keys recommended, single key, automatic routing across providers.
• console.anthropic.com/settings/keys
• platform.openai.com/api-keys
• aistudio.google.com/apikey

Permissions AIPointer needs

AIPointer asks for the minimum required to do its job. Nothing else is touched.

macOS

• Accessibility (required) — lets AIPointer detect the global trigger hotkey from inside any app. Without this, the hotkey is dead. Grant in System Settings → Privacy & Security → Accessibility.
• Screen Recording (required) — capture the small region around your cursor for vision context. Triggers only on hotkey hold, never in the background.
• Microphone (optional) — needed only for voice input or the voice-first conversation loop. Recordings stream to the AI provider you configured and are never stored locally.
• Automation → Finder & System Events (optional) — needed only for the Finder auto-attach workflow (select files → press hotkey → files attach). On first use macOS asks "AIPointer wants to control Finder" → pick OK. If you click Deny or dismiss it, the trigger shows an amber banner explaining how to re-enable in System Settings → Privacy & Security → Automation → AIPointer → Finder. Without it, the paperclip button still works for manual attachment.

Windows

• No special permissions at install. Microphone prompted on first use when needed.
• Explorer auto-attach walks Shell.Application via PowerShell. Some Defender configurations block this — it degrades gracefully to the paperclip button.

Linux

• libsecret (gnome-keyring) or KWallet for OS-encrypted API key storage. Without one, keys store base64-encoded with a warning banner.
• File-manager auto-attach is not supported (no universal selection mechanism across DEs). Paperclip button works.

All platforms — network access Your prompts, screenshots, queued files, and voice recordings (if used) go directly to the LLM provider you configured. The auto-updater fetches aipointer.app/updates/latest.json. Nothing else leaves your machine. No telemetry, no analytics.

Voice (TTS + STT) — 3 engines, System by default

AIPointer ships a 3-state Voice engine picker in Settings → Voice. Fresh installs land on System so read-aloud and voice input work for 100% of users without any setup. Local and Cloud are explicit opt-ins.

The Local engine, in detail. Settings → Voice → Local discloses the download size before pulling anything and shows live progress; atomic writes prevent a partial file from