Features at a Glance
OpenGUI uses real screenshots to understand apps and carry out phone operations and testing tasks from natural-language instructions.
- Phone operations: open apps, tap, swipe, type, navigate back, and move between screens to complete multi-step workflows.
- App testing: check features and workflows, record issues with reproduction steps and screenshots, and summarize results.
- Content and information: read screen content, organize information, fill forms, and prepare social media drafts.
- Visible execution: use the WorkBuddy workbench to view the phone screen, task steps, and reports, or take over and stop a task.
- Host integration: use OpenGUI in WorkBuddy or DeepSeek Harness; the DSH plugin also supports managed browser operations.
Common Commands
After installation and authorization, enter these instructions in the corresponding host's chat. Replace [placeholders] with your task details.
| Purpose | Host | Example input |
|---|---|---|
| Open OpenGUI | WorkBuddy | /opengui |
| Check connected devices | WorkBuddy | /opengui List connected devices without operating a phone. |
| Operate a phone | WorkBuddy | /opengui Open Settings, then return to the home screen. Stop after confirming it is visible. |
| Test an app | WorkBuddy | /opengui Test [feature or workflow] in [app/page]. Record issues, steps, and screenshots. Stop at [end condition]. |
| Prepare a Tiktok draft | WorkBuddy | /opengui Prepare a Tiktok image-and-text draft about [topic]. Stop before publishing. |
| Run a task in DSH | DeepSeek Harness | @OpenGUI Open Settings and report the Android version |
Use OpenGUI in WorkBuddy
Supported on macOS (Apple Silicon / Intel) with WorkBuddy 5.5.3 or later. The current public release is 0.4.0, a public-testing prerelease. Download and run the installer on your Mac, restart WorkBuddy once, then authorize OpenGUI.
[!IMPORTANT] Restart WorkBuddy once after installation. Finish other tasks, fully quit with ⌘Q, then reopen WorkBuddy before authorizing OpenGUI. Closing its window is not enough.
Step 1. Install with a Terminal command
Open macOS Terminal, paste this command, and run it:
cd "$HOME" && curl -fsSL https://raw.githubusercontent.com/Core-Mate/OpenGUI/main/workbuddy-plugin/install.sh | bash
This downloads and runs the installer without saving it manually first. The script verifies the subsequent installation files, prepares the runtime, and configures OpenGUI.
Node and npm dependencies default to the China npmmirror service with official fallback, scoped to this installation. To use official sources, replace the final bash with bash -s -- --download-source official. Video assets use GitHub separately; an npm mirror does not accelerate them. See download source options for a custom video mirror.
You can also download install.sh separately and run bash /actual/path/install.sh in Terminal.
Step 2. Restart WorkBuddy once
After the installer finishes, complete other WorkBuddy tasks, fully quit with ⌘Q, and reopen WorkBuddy to load the newly installed MCP configuration.
Step 3. Authorize the Skill and connector
In WorkBuddy, open Experts · Skills · Connectors → Skills, find OpenGUI, and complete authorization. Then open Connectors, find OpenGUI, and authorize it as well.
Both the Skill and connector require authorization.
If OpenGUI is not listed under Connectors, check Custom connectors for its MCP service. If the authorization page did not open automatically, open the AUTHORIZATION_GUIDE path printed in Terminal; no reinstallation is needed.
Step 4. Verify the connection and start using OpenGUI
Enable USB debugging on your Android phone, connect it to your computer, and approve USB debugging on the phone. Enter /opengui, then enter the instructions you want the phone to execute to control it.
Try it: Post on Tiktok
/opengui Prepare a Tiktok image-and-text draft about [topic] using [materials]. Stop before publishing.
Try it: Vibe Testing
/opengui Help me test [feature or workflow] in [app/page], focusing on [issues to check]. If you find any problems, record the steps and screenshots. Stop when [end condition] is reached.
Use OpenGUI in DeepSeek Harness
The shortest path on macOS is to let Codex run the stable installer Skill from main. Each run resolves the latest stable OpenGUI plugin release, while an explicit version remains available for rollback. It requires Node.js 22.19+ or 24+ and installs the compatible DSH version automatically. Paste this as one prompt:
Install and run the OpenGUI installer Skill from https://github.com/Core-Mate/OpenGUI/tree/main/deepseek-harness-plugin/skills/opengui-coremate-install for my DSH web profile. Install the latest stable release. Proceed autonomously, and only pause when I need to authorize or select a phone, add or select a DSH workspace, or provide fallback visual-model credentials.
The Skill downloads the public release package and checksum, verifies SHA-256, installs only the OpenGUI plugin, starts DSH when needed, and opens DSH. It preserves unrelated DSH plugins and settings. The installer reports whether it reloaded a managed DSH or whether you need to quit an existing process and rerun it. For Linux or Windows, use the manual package guide.
Supported DSH versions are defined in the installer compatibility list. The default is 0.1.1-rc.2; use --dsh-version VERSION to select a listed version. The installer preserves other plugins, workspaces, model settings, and credentials. See the plugin installation guide for version changes and rollback constraints.
After installation, add or select a DSH workspace, connect and select an authorized Android phone, then send:
@OpenGUI Open Settings and report the Android version
The plugin adds phone and browser operation to DSH without requiring the full OpenGUI backend stack. The current source implementation admits one OpenGUI task per DSH session and separate tabs on non-conflicting phone sets; the managed browser remains globally serial. This source behavior is not a release claim. See more use cases or download the v0.1.13 release package.
GUI execution requires a model with image input and tool calling. DSH uses the current session model by default; configure a fallback visual model when needed, following the plugin guide. Evaluate execution quality, latency, and actual costs on your own tasks.
Run the Full OpenGUI Stack
Use the full stack to self-host the backend and Android client or dispatch tasks through Feishu, Telegram, Discord, or REST APIs. WorkBuddy and DSH plugin users do not need this deployment.
Open Claude Code, Codex, or OpenCode in the repository root and send:
Read ./skills/open-gui-bootstrap/SKILL.md and help me run OpenGUI. Only ask me for phone-side actions.
The full-stack Android client requires Android 11 (API 30) or later, USB debugging, AccessibilityService, overlay permission, and a battery optimization exemption. See the deployment guide and Android permission guide for model configuration, manual startup, and phone authorization.
After deployment, run the CLI from the server directory:
pnpm opengui -- devices --json
pnpm opengui -- do "Observe the current Android screen, summarize what you see, and stop" --json
pnpm opengui -