nuphus-mcp
Desktop automation MCP server — computer use for any AI agent. See the screen, control windows/mouse/keyboard, and drive Chrome over the Model Context Protocol (stdio). Desktop & browser automation need no API key; OCR runs locally; vision plugs into your own vision LLM (OpenAI-compatible or Anthropic native, BYOK).
nuphus-mcp is a lightweight, cross-platform desktop automation MCP server
that exposes desktop + browser automation as standard MCP tools. It speaks
JSON-RPC 2.0 over stdio — no daemon, no network service, one binary. Claude
Desktop, Cursor, VS Code, Copilot, or any MCP client can connect and
immediately control the screen, windows, keyboard/mouse, and Chrome —
computer use for any AI agent — desktop & browser automation need no API
key; local OCR is built in; vision works with your own vision LLM
(OpenAI-compatible, BYOK).
🇨🇳 Mainland China mirror: this repo is mirrored on Gitee for fast in-China access (Chinese docs served by default there). 中文文档
Part of the Nuphus ecosystem: Nuphus — a local-first AI agent with real desktop execution and dual-device (phone-as-second-screen) sync. nuphus-mcp exposes the same desktop + browser automation to any MCP client.
Using DeepSeek Harness (DSH)? nuphus-mcp itself is a plain stdio MCP server — not a DSH/Cordis plugin. Install the dedicated dsh-nuphus-mcp plugin instead (see DeepSeek Harness (DSH)).
┌──────────────────┐ stdio JSON-RPC ┌──────────────────────┐
│ Any MCP Client │ ───────────────► │ nuphus-mcp │
│ (Claude/Cursor/ │ ◄─────────────── │ desktop-api crate │──► screen/window/mouse/keyboard
│ Nuphus itself) │ single-line JSON │ nuphus-browser crate│──► Chrome (CDP)
└──────────────────┘ └──────────────────────┘
Features
- 38 MCP tools (15 desktop + 23 browser) — screenshots, window control, mouse/keyboard, Chrome CDP automation, and more — see TOOLS.md / TOOLS.zh-CN.md for the full reference.
- Desktop automation: screen size, screenshot (PNG/base64), window list,
window activate/screenshot/move/resize/info, mouse click/drag/scroll/position,
keyboard input/hotkey, clipboard write/clean — implemented on the
desktop-apicrate (xcap + Win32, no Tauri dependency). - Computer vision pair:
desktop_vision(BYOK — send a screenshot to your own vision model via an OpenAI-compatible or Anthropic native API) +desktop_perceive(local OCR with PaddleOCR, models auto-downloaded on first run; optional YOLO icon detection). Used together they give AI agents both semantic understanding and pixel-precise coordinates — the battle-tested vision→perceive flow from the Nuphus desktop app. See TOOLS.md for BYOK env vars, model setup, and the recommended flow. - Browser automation: navigate, snapshot (accessibility tree with
@Nrefs), click, type, exec, scroll, extract, screenshot, evaluate, back/forward, wait_for, cookies get/set/import, upload, tabs, downloads — implemented onnuphus-browser(chromiumoxide CDP). - Zero-cost stdio: no HTTP server, no daemon. The process reads single-line JSON from stdin and writes responses to stdout.
- Safety-first: destructive tools are annotated per the MCP spec; optional strict-confirm mode; path validation for screenshots, uploads, and file drags.
Repository Layout
nuphus-mcp/
├── Cargo.toml # workspace root
├── TOOLS.md / TOOLS.zh-CN.md # 38-tool reference
├── crates/
│ ├── nuphus-mcp/ # MCP Server (this repo's product)
│ ├── nuphus-browser/ # Browser automation core (CDP)
│ └── desktop-api/ # Desktop control core (vendored)
└── ...
Prerequisites
- Rust toolchain (stable) — build from source with Cargo.
- Chrome or Edge — required for browser tools. The server auto-detects an
installed browser; if none is found,
browser_*tools return a clear error. - Windows recommended for full desktop control — see Platform Support below.
Platform Support
| Platform | Browser tools | Desktop tools |
|---|---|---|
| Windows | Full | Full (Win32 API) |
| macOS | Full | Desktop input requires Accessibility permission (System Settings → Privacy & Security → Accessibility) |
| Linux | Available | Partial — window/input capabilities are limited |
Execution HUD: non-intrusive execution feedback on every platform — Windows shows an on-screen OSD bar (start + result, real-time), macOS/Linux post a system notification on completion (
NUPHUS_MCP_HUD=offto disable). Window activation is never used as a visibility fallback.
API Keys & Local Models
Vision — BYOK, OpenAI-compatible or Anthropic native
desktop_vision uses your own vision model. It speaks two protocols:
- OpenAI-compatible Chat Completions (default) — works with OpenAI, MiniMax, Qwen, Ollama, vLLM, …
- Anthropic native Messages API — point
NUPHUS_MCP_VISION_BASE_URLathttps://api.anthropic.com/v1and the protocol is auto-detected from the host; or force it withNUPHUS_MCP_VISION_PROVIDER=anthropic.
Nothing is required unless you call this tool — and when it is not configured the tool returns a clear error instead of silently failing.
| Environment variable | Required | Default | Description |
|---|---|---|---|
NUPHUS_MCP_VISION_API_KEY |
✅ | — | API key for your vision model |
NUPHUS_MCP_VISION_BASE_URL |
— | https://api.openai.com/v1 |
Base URL (https://api.anthropic.com/v1 for Claude) |
NUPHUS_MCP_VISION_MODEL |
✅ | — | Model id, e.g. gpt-4o-mini, qwen-vl-max, claude-sonnet-4-5 |
NUPHUS_MCP_VISION_PROVIDER |
— | auto |
auto | openai | anthropic; auto infers from the base URL host |
NUPHUS_MCP_VISION_MAX_TOKENS |
— | 1024 |
Max output tokens (Zhipu GLM-4V-Flash caps at 1024; raise for text-heavy screenshots) |
External browser (anti-detect / fingerprint browsers)
By default browser_* tools launch and manage their own Chrome instance. To
drive an external browser instead — e.g. an anti-detect / fingerprint
browser — start it with a debugging port and point the server at it:
| Environment variable | Required | Default | Description |
|---|---|---|---|
NUPHUS_MCP_BROWSER_CDP_URL |
— | — | External CDP endpoint, e.g. http://127.0.0.1:9222 |
# Example: start your fingerprint browser with a debugging port
chrome --remote-debugging-port=9222 --user-data-dir=...
// MCP client config
"env": { "NUPHUS_MCP_BROWSER_CDP_URL": "http://127.0.0.1:9222" }
When set, browser_* tools attach to that endpoint and never launch a managed
Chrome; attach failures are hard errors (no silent fallback into the wrong
browser). The external browser belongs to you — the server never kills it on
exit.
Self-healing on window reopen (recommended)
Fingerprint browsers typically get a new random debug port every time a
window is reopened, which would leave a fixed ..._CDP_URL pointing at a
dead port. Provide the browser identity and the server re-resolves the live
port automatically on the next tool call:
| Environment variable | Required | Default | Description |
|---|---|---|---|
NUPHUS_BROWSER_EXE_PATH |
for self-healing | — | Absolute path to the browser exe — the identity key used to locate the running window process |
NUPHUS_BROWSER_NAME |
— | 指纹浏览器 |
Display name used in error guidance |
NUPHUS_BROWSER_USER_DATA_DIR |
— | — | Profile dir, used to read DevToolsActivePort when the window was started with a random port (--remote-debugging-port=0) and the process cmdline does not reveal it |
// MCP client config — attach + self-heal
"env": {
"NUPHUS_MCP_BROWSER_CDP_URL": "http://127.0.0.1:9222",
"NUPHUS_BROWSER_EXE_PATH": "C:\\path\\to\\fingerprint-browser.exe",
"NUPHUS_BROWSER_NAME": "AdsPower",
"NUPHUS_BROWSER_USER_DATA_DIR": "C:\\path\\to\\profile"
}
With the identity set, if the configured endpoint stops answering the server
locates the window process by exe path, re-resolves its actual port (literal
cmdline port, or DevToolsActivePort in the profile dir for random-port
launches), verifies the candidate with a proxy-bypassing CDP probe and retries
once — close and reopen the window and the next tool call just works. Without
an identity, attach failures stay hard errors asking you to update the
configured URL. Either way there is no fallback to a managed Chrome.
Keeping logins: drive a browser that has your extensions & sessions
Want browser_* tools to drive a browser carrying your own extensions /
bookmarks / login state? First, a hard Chrome 136+ restriction: after
Chrome's official security change, --remote-debugging-port and
--remote-debugging-pipe are ignored against the default user data
directory — they must be paired with --user-data-dir pointing at a
non-default directory. This is a deliberate design to stop infostealer ma