Clicky for Windows 🔵

An AI teaching companion that lives next to your cursor. Ask it anything about your screen — it points, explains, and guides you step-by-step, like a real tutor sitting beside you.

Clicky is a Windows port of farzaa/clicky (originally macOS/SwiftUI). Built with Python 3.11+ and PyQt6, runs fully in the background, and works with every major AI provider.

Hi, this is Clicky 👋

🎬 Watch full demo on YouTube →

Clicky is a little AI buddy that lives next to your cursor. Hold a hotkey, ask it something about your screen, and it talks back — pointing at buttons, walking you through steps, drawing arrows on your screen. Think of it as a patient tutor sitting beside you while you learn anything: video editing, coding, a new app, whatever.

No more Alt-Tab to ChatGPT. No more typing out descriptions of what's on your screen. Just hold Ctrl + Win, speak, and Clicky handles the rest.

Works 100% offline with Ollama or LM Studio, or plug in your Claude / OpenAI / Gemini / GitHub Copilot key for the full experience.

What it looks like

┌─────────────────────────────────────────────┐
│  Your screen (browser, IDE, Premiere, etc.) │
│                                             │
│          🔵◂ ← Clicky, your blue buddy      │
│          (floats beside your real cursor)   │
│                                             │
│  ┌───────────────────────────┐              │
│  │  Clicky  [Claude]     —   │  ← panel     │
│  │  ● Thinking…              │              │
│  │  "The search bar is       │              │
│  │   right here ↗"           │              │
│  │  Model: claude-sonnet-4…  │              │
│  └───────────────────────────┘              │
└─────────────────────────────────────────────┘

The blue triangle sits 35 px right / 25 px below your real cursor. When you ask it to point at something, it flies there along a smooth bezier arc (teacher pace), dwells with a pulsing highlight ring, then flies back.

Feature List

🎙️ Voice Activation

• Hold Ctrl + Win to push-to-talk
• Say "Clicky" for hands-free wake word
• Press Esc to stop any response or TTS mid-stream

👁️ Screen Aware

• Full multi-monitor screenshot on every query
• Describes only what it sees — never hallucinates
• Detects active window title for per-app memory

🎯 Pixel-Perfect Pointing

• Two-stage grid locator works with any vision LLM — Claude, Copilot, OpenAI, Gemini, Ollama, LM Studio
• Stage 1: draws a numbered 12×8 grid on the screenshot, the LLM picks a cell
• Stage 2: zooms into that cell and runs a 6×6 fine-grid pass for sub-cell precision
• Bezier arc flight with configurable teacher pace
• Pulsing highlight ring + speech bubble label on the target
• Works on any DPI scale (4K, HiDPI, multi-monitor setups)

🧑‍🏫 Real Tutor Behaviour

• Locate queries → points + 1-sentence explanation (no generic text directions)
• Multi-step tasks → breaks into steps, says "Say 'next' when ready"
• "Next" / "Continue" → advances the lesson without a new LLM call
• "Repeat" → replays the last answer via TTS without re-querying
• "Stop" / "Cancel" → cancels generation instantly

🌐 Web Search (Real-Time Data)

• DuckDuckGo HTML scrape + concurrent page fetch — no API key needed
• Optional Tavily upgrade for deeper results
• Responses grounded in today's data with [1], [2] citations
• Always knows today's date — never gives stale answers

🔄 Multi-Provider LLM (Runtime Switching)

Priority chain (auto-detected): Claude → OpenAI → Copilot → Gemini → Ollama / LM Studio

Switch mid-session from the system tray — takes about a second, no restart needed.

📋 Live Model Lists (Auto-Refreshed)

• Claude, OpenAI, Gemini: live model list fetched from vendor APIs, 30-day cache
• GitHub Copilot: live /models endpoint with billing multiplier, 6-hour cache
• Free Copilot models auto-prioritised (no premium quota burned by default)
• Panel model dropdown always reflects the actual models available to your account

🔊 Multi-Provider TTS

🗣️ Multi-Provider STT

🧠 Per-App Memory

• Separate conversation history per active application
• Context never bleeds between Premiere, VS Code, Chrome, etc.
• History cap: last 20 messages per app

📝 Knowledge Journal + Spaced Repetition

• Every Q&A automatically logged to a local SQLite database
• Ask "what did we cover today?" → summary of today's session
• Ask "what did we cover this week?" → weekly digest
• SM-2 spaced repetition — Clicky reminds you to review topics at optimal intervals (1 → 3 → 7 → 14 → 30 → 60 → 120 days)
• Say "quiz me on what I should review" → flashcard session from your journal
• Journal stored at %LOCALAPPDATA%\Clicky\journal.db

📄 Document Context (Drag & Drop)

• Drag a PDF, DOCX, TXT, MD, CSV, or code file onto the Clicky panel
• Clicky reads it and uses it as context for your next questions
• Or use Tray → Journal → Attach document… for a file picker
• Supports multi-file: attach several docs and ask cross-document questions

🔍 OCR Fallback (Fine Print)

• When your query mentions "fine print", "small text", "read that", etc., Clicky runs Tesseract OCR on the screenshot
• Extracts text too small or low-contrast for the LLM's vision to read
• Requires the Tesseract binary from UB-Mannheim/tesseract
• Gracefully skipped if Tesseract isn't installed

💻 Code Mode (Auto-Detect)

• Detects VS Code, Cursor, IntelliJ, PyCharm, Vim, Neovim, Sublime, etc. by window title
• Automatically injects a code-specialist system prompt addendum: prefers code blocks, includes language tags, explains algorithms step by step
• Toggle: Tray → Tutor Mode → Code Mode (auto)

🎓 Tutor Modes (Tray → Tutor Mode)

🌍 Multilingual Auto-Detect

• Detects query language via langdetect + Unicode script heuristics
• Responds in the same language automatically
• Switches the Edge TTS voice to match (Hindi → HiRA-Neha, French → fr-FR-DeniseNeural, etc.)
• 15+ languages supported: EN, HI, FR, DE, ES, PT, AR, ZH, JA, KO, RU, IT, NL, PL, TR
• Toggle: Tray → Tutor Mode → Multilingual

🎬 Lesson Recording (MP4 + Transcript)

• Records the screen at 8 fps as an MP4 alongside a Markdown transcript of all Q&A
• Start: Tray → Lesson Recording → Start recording
• Stop: tray menu → produces lesson_YYYY-MM-DD_HH-MM.mp4 + lesson_YYYY-MM-DD_HH-MM_transcript.md
• Saved to %LOCALAPPDATA%\Clicky\recordings\

🖊️ Whiteboard Annotations

Clicky can draw directly on your screen as it explains:

• [ARROW:x1,y1->x2,y2] — animated arrow between two points
• [CIRCLE:x,y,r:label] — pulsing circle with optional label
• [UNDERLINE:x,y,w] — underline beneath text
• [LABEL:x,y:text] — floating text label

Annotations fade out after ~4 seconds and clear automatically on the next query.

🖱️ Workflow Capture

• Records your mouse clicks and keystrokes while you work
• Start: Tray → Workflow Capture → Start capturing my clicks
• Stop: Tray → Workflow Capture → Stop + send to Clicky
• Ask "What did I just do?" → Clicky narrates your workflow step-by-step
• Requires pynput (included in requirements.txt)

🎤 Voice Picker

• Switch TTS voice mid-session: say "change voice to [voice name]"
• Edge TTS voices: AvaNeural, JennyNeural, GuyNeural, AriaNeural, etc.
• List all 400+ voices:

  python -c "from audio.tts.edge_tts_provider import EdgeTTSProvider; print(EdgeTTSProvider.list_voices_sync())"
  

🔒 Privacy Guard (on by default)

Detects sensitive windows and skips the screenshot entirely:

• Password managers: KeePass, Bitwarden, 1Password, LastPass, Authenticator
• Sensitive pages: login, sign in, banking, .env files, credit card

🛠️ Skills System (User-Extensible)

Create your own voice triggers — no pull request needed:

# ~/.clicky/skills/my_skill.py
import asyncio

SKILL = {
    "name": "open_calculator",
    "trigger": r"(open|launch|start) calculator",
    "description": "Opens Windows Calculator",
    "handler": open_calc,
}

async def open_calc(transcript: str, manager) -> str:
    import subprocess
    subprocess.Popen("calc.exe")
    return "Opening calculator for you."