GhostDesk

One binary, one MCP endpoint, three desktops. The tool surface is the same everywhere — an agent calls mouse_click and app_launch without knowing which desktop it is on.

Linux is the server. The container is the deployment this README shows unless it says otherwise, and the only one of the three with a sandbox around it. macOS and Windows run the very same server against a real desk — no container, and therefore no isolation. Jump to macOS or Windows, or read what differs between the three.

• Quick start
• Tools
• Model requirements
• Why GhostDesk?
• How it works
• Secure local run (TLS + auth)
• Running many agents
• Custom image
• Run on macOS, without the container
• Run on Windows, without the container
• What differs from the container
• Configuration
• Security
• Troubleshooting
• Build from source
• License

Quick start

1. Run the container

One command, plain HTTP, no password. Fine for kicking the tires on a laptop you trust — not fit for anything beyond that. Ready to harden it? Jump to Secure local run.

docker run -d --name ghostdesk-demo \
  --shm-size 2g \
  -p 3000:3000 \
  -p 6080:6080 \
  ghcr.io/yv17labs/ghostdesk:latest

The latest image ships with Firefox, the foot terminal, mousepad (text editor), galculator, and passwordless sudo for the agent user — enough to demo a browsing + note-taking workflow out of the box. Need a different app set? Build your own on top of base — see Custom image.

The container boots in the dev posture: plain HTTP on both ports, every auth gate disarmed on purpose. You'll see warnings in the logs reminding you of that — they go away once you follow the secured path below.

2. Connect your agent

GhostDesk speaks MCP over the Streamable HTTP transport — any MCP-compatible client can drive it. Point the one you already work from at http://localhost:3000/mcp: no headers, no auth, that is the whole demo posture.

Claude Code / Claude Desktop

{
  "mcpServers": {
    "ghostdesk": {
      "type": "http",
      "url": "http://localhost:3000/mcp"
    }
  }
}

Hermes Agent, OpenCode, or whatever you already drive your agents with — same URL, and it works very well.

SpecterChat — the chat client we build for this, open source. Most chat UIs drop the image an MCP tool returns — they render it or they forward it to the model, rarely both — and a screen_shot() the model never sees is the whole product missing. SpecterChat displays it inline and sends it back as base64. It talks to any OpenAI-compatible endpoint (llama.cpp, vLLM, Ollama, LM Studio), so it takes either side of the next step; macOS, Linux and Windows builds are on its releases page.

3. Choose where the model runs — cloud or local

The client is settled; the model behind it is a choice, and neither answer is mandatory.

In the cloud — Claude, GPT, Gemini and the other frontier models drive GhostDesk out of the box: no header, nothing to install, the key your client already holds. If that is you, the setup is done — go to step 4.

On your machine — Ollama is the recommended path: one installer, one model to pull, nothing to build and nothing to tune. Windows, macOS or Linux. No API bill, and no screenshot of your desktop leaving your network.

1. Install Ollama from ollama.com/download — the official app, downloaded like any other. Once installed it runs in the background, so there is no server to start.

2. Pull the model. In a terminal:

   ollama pull qwen3.8:27b
   

   On Apple Silicon, pull qwen3.8:27b-mlx instead.

3. Point your client at it. Endpoint http://localhost:11434/v1, the tag you pulled as the model name, and the GhostDesk-Model-Space: 1000 header on every MCP request — the Qwen family needs it, and the JSON for it is under Coordinate space.

That is the whole setup. qwen3.8:27b is the default because it is the strongest — best reasoning, best capability, and on desktop control a step the agent gets wrong costs more than the tokens it saves. When speed is what you need, qwen3.6:35b (on Apple Silicon, qwen3.6:35b-mlx) is the fast one: 35B parameters with only 3B active per token (Qwen3.6-35B-A3B), so tokens per second is what you feel. All four tags are tested with GhostDesk, and the verdict is the same on every backend on this page. Want the flags in your own hands? That is Your own inference server.

4. Watch your agent work

Open http://localhost:6080/ in your browser to see the virtual desktop in real time. No password prompt — the dev posture skips it.

The probes are what the container's HEALTHCHECK reads, and they answer about the desktop rather than about the processes: ready goes down when the compositor stops answering the window seam, live when the connection behind the virtual pointer and keyboard is gone — a state in which every tool still replies and none of them does anything.

Give your agent a first prompt to confirm the wiring is right:

"Take a screenshot of the desktop, list the installed applications, then open Firefox and go to wikipedia.org."

You should see Firefox launch in the noVNC tab, the URL bar fill in, and the page load — all under your agent's control.

5. When you're done

docker stop ghostdesk-demo && docker rm ghostdesk-demo

The demo run creates no named volume, so this leaves nothing behind.

Tools

Fourteen tools, named verb_noun, and this is the whole surface — no hidden endpoint, no second protocol. Defaults are in parentheses, ? marks an optional parameter, and every coordinate is a pixel offset in the last screen_shot() — measured from that capture's own corner, so a capture taken with a region is answered in coordinates that only mean anything when the same region travels to the act using them. The moment the screen changes, coordinates computed from the previous capture are stale.

Screen