Ghostwork
The first agent you don't prompt.
Ghostwork runs silently in the background, learns how you work from observation alone, and gradually takes repetitive tasks off your hands — without you ever writing a prompt or setting up an integration.
Free Screenpipe GUI
Screenpipe's desktop app moved to a paid model ($25/month). Ghostwork works with the free Screenpipe CLI and provides a full open-source GUI layer on top:
| Feature | Ghostwork (free) | Screenpipe desktop ($25/mo) |
|---|---|---|
| Timeline view | ✅ | ✅ |
| Natural language search | ✅ | ✅ |
| App usage analytics | ✅ | ✅ |
| macOS workflow automation | ✅ | ❌ |
| Learns from observation | ✅ | ❌ |
| Runs autonomously | ✅ | ❌ |
| Local / no cloud | ✅ | ✅ |
| Open source | ✅ GPL-3.0 | Source-available |
Ghostwork does everything the paid Screenpipe GUI does, plus it automates the workflows it discovers — for free.
What it does
Ghostwork watches every interaction across every app on your Mac (via Screenpipe), builds a rich memory of your work patterns, and surfaces automations the moment they're relevant.
Memory layers
| Layer | What's stored | How it's built |
|---|---|---|
| L1 Working | Current screen context (app, URL, OCR) | Live, polled every 10 s |
| L2 Episodic | Raw interactions: clicks, keys, navigations, app switches | Ingested every 2 min from Screenpipe's input stream |
| L3 Semantic | Workflows and rules: "WHEN on LinkedIn search → DO export to CRM" | Promoted nightly from episodic memory via LLM |
| L4 Procedural | Executable skills with step-by-step DOM/AX locators | Promoted nightly from stable semantic rules |
Nightly consolidation (sleep cycle)
Every night Ghostwork runs a 3-phase consolidation:
- NREM — LLM analyses unsummarised sessions and promotes patterns to rules
- REM — Rules with 3+ observations that have browser-recorded events are compiled into executable skills
- GC — Power-law confidence decay, dedup, 90-day prune of raw events,
behaviour.mdrewrite
The living behaviour.md profile is injected into every LLM prompt, so the trigger decision has full context about who you are and what you do.
Autonomy tiers
Ghostwork earns autonomy, never assumes it:
| Tier | Triggered when | Behaviour |
|---|---|---|
| Supervised | Default for all new rules | Executes immediately, shows HUD notification, Cmd+Z available |
| Autonomous | ≥5 accepts and <2 rejections in last 10 | Runs silently; logged to Activity feed |
Actions that are externally visible (send email, submit form, post) always require one-tap approval regardless of tier.
Execution stack
- Compiled skill replay — zero-token deterministic replay of recorded step sequences
- AX-first native control —
ax_list_elements+ax_click_elementvia macOS Accessibility API; ~95% accuracy on native apps - Claude vision fallback — pixel-level screenshots + function calling for browsers and AX-empty apps
When a multi-step sequence fails mid-way, Ghostwork fires Cmd+Z for each completed reversible step in reverse order before surfacing the error.
UI
- Menu bar icon — reflects current state: observing / working
- Activity feed — chronological log of every action taken and approval waiting
- Timeline tab — drill down into every recorded session; save any session as a skill
- Behaviour tab — learned rules with autonomy progress, category badges, and one-click boost
- Approvals — staged actions waiting for your confirmation before executing externally visible steps
Setup
Prerequisites
- macOS 12+
- Screenpipe installed and running (
screenpipe) - Node.js 20+ and npm
- An OpenRouter API key (or Anthropic key)
Install
git clone https://github.com/your-org/ghostwork
cd ghostwork
npm install
npx @electron/rebuild -f -w better-sqlite3
Configure
cp .env.example .env
Edit .env:
OPENROUTER_API_KEY=sk-or-...
ANTHROPIC_API_KEY=sk-ant-... # optional, used for native computer use
Run
npm start # development (hot reload)
npm run build # production build
npm run dist # package as .dmg
Architecture
Screenpipe ──► sessionIngester (2min) ──► raw_events ──► extractor (30min) ──► rules
│
NREM/REM nightly
│
compiled skills
│
actionEngine (10s) ──► LLM trigger ──► AX-first executor
└──► vision fallback
Key files:
| File | Responsibility |
|---|---|
sessionIngester.ts |
2 min poll → raw_events + per-event prediction scoring |
extractor.ts |
30 min batch → 7-category structured rule extraction |
consolidation.ts |
Nightly NREM (sessions→rules) + REM (rules→skills) + GC |
actionEngine.ts |
10 s perception loop + LLM trigger decision + dispatch |
computerUse.ts |
AX-first executor + Claude vision fallback |
axDriver.ts |
macOS accessibility tree (AXUIElement via AppleScript) |
skillEngine.ts |
Browser skill replay with multi-step rollback |
approvals.ts |
Shadow-mode approval queue for externally visible actions |
db.ts |
GhostWork SQLite: rules, episodes, skills, approvals, settings |
Contributing
See CONTRIBUTING.md for setup instructions and coding guidelines.
Open issues on GitHub — look for good first issue labels. Join the Discord to discuss ideas or get help getting set up.
Privacy
- All data stays on your device. No cloud sync, no telemetry.
- Ghostwork excludes itself, Cursor, and any app you add to the exclusion list.
- Raw events are pruned after 90 days.
- PII (emails, phone numbers, card numbers) is stripped before any LLM call.
- The
behaviour.mdprofile never leaves your machine.
Roadmap
- Full autopilot mode: skill execution without any prompt
- Cross-session pattern detection in REM phase
- Browser extension for richer DOM locators in user's main Chrome profile
- Windows support (via Screenpipe Windows builds)
- Team profiles (opt-in, anonymised)
Licence
GPL-3.0 — see LICENSE for details.