Maintainer of this skill?
by TIGER-AI-Lab
Open-source benchmark for browser AI agents on daily tasks.
# Add to your Claude Code skills
git clone https://github.com/TIGER-AI-Lab/ClawBenchGuides for using ai agents skills like ClawBench.
See how ClawBench compares with popular alternatives.
ClawBench is an open-source ai agents skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by TIGER-AI-Lab. Open-source benchmark for browser AI agents on daily tasks. It has 1,212 GitHub stars.
ClawBench's catalog security scan is still queued. You can run an instant dependency and prompt-injection check now with the "Scan for vulnerabilities" button above.
Clone the repository with "git clone https://github.com/TIGER-AI-Lab/ClawBench" and add it to your Claude Code skills directory (see the Installation section above).
ClawBench is primarily written in Python. It is open-source under TIGER-AI-Lab on GitHub, so you can review or fork the full source.
Yes. SkillsLLM lists many other AI Agents skills you can browse and compare side by side. Open the AI Agents category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh ClawBench against similar tools.
No comments yet. Be the first to share your thoughts!
Available after the catalog security scan passes.
⚠️ Third-Party Software Notice
This skill is third-party open-source software developed and hosted independently on GitHub. SkillsLLM is an informational directory and does not control or maintain the underlying repository.
Any security checks, ratings, or warnings displayed by SkillsLLM are automated and limited in scope. They do not constitute a security certification or guarantee that the software is safe, error-free, or free from malicious code, vulnerabilities, compromised dependencies, or prompt-injection risks.
Review the source code, permissions, dependencies, and configuration before installing or running any third-party skill. Use is at your own risk. To the maximum extent permitted by applicable law, SkillsLLM is not liable for losses arising from third-party software.
The deep catalog scan for this skill is still queued. Run an instant dependency check now instead.
Quick start · Results · Docs · Citation · 中文 · WebsiteBench
ClawBench is an open-source benchmark, published in Findings of EMNLP 2026, that evaluates AI browser agents on everyday online tasks — booking travel, ordering food, applying for jobs, managing email — across live websites. It records five layers of evidence and supports request-based scoring and trajectory evaluation against human references. In the historical V1 paper evaluation, the strongest evaluated agent completed about 1 in 3 tasks.
V1: 152 tasks · 143 live sites | V2: 129 tasks · 63 live sites | 281 tasks across 163 live websites · 15 life categories
Built by NAIL Group · Sister project: WebsiteBench — Evaluation of agents' ability in reverse-engineering websites · Runs on any Chrome.
Featured in 37 curated lists and #3 Paper of the Day on Hugging Face: docs/featured.md.
| I want to… | Start here |
|---|---|
| Run my model | Quick start |
| See scores | Live leaderboard · Results |
| Browse the tasks | Task explorer · Datasets |
| Reproduce a score | Reproduction workflow · Trace cookbook |
| Understand the benchmark | Paper · FAQ · How ClawBench compares |
| Change the framework or add tasks | Source layout · Contributing |
All guides and integrations: Documentation index.
Prerequisites: Python 3.11+, uv, and a running Docker or Podman engine. See platform setup.
Install clawbench-eval from PyPI:
uv tool install clawbench-eval
clawbench
Launch from the directory where you want your configuration and results.
The interactive TUI guides you through adding a model and selecting a task.
Configure the default judge, deepseek-v4-pro, as well; the
model setup guide explains judge
selection and how to run without scoring.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git
cd ClawBench
cp models/models.example.yaml models/models.yaml
# Edit models/models.yaml with your model and judge API settings.
uv run clawbench
./run.sh is a shortcut for the same TUI. For a non-interactive run from the
checkout:
uv run clawbench-run test-cases/v1/886-entertainment-hobbies-experience-topgolf your-provider-model-id
Replace your-provider-model-id with a configured model key. The first run
builds container images; the local browser's printed noVNC URL lets you watch
the agent. Recordings and metadata land under test-output/.
Continue with the getting started guide for
batch and human runs, harness selection,
and remote browser options. The CLI reference covers
non-interactive commands. Coding agents can start with AGENTS.md.
You pick a task ClawBench spins up Agent drives the Interceptor captures
from V1 or V2 an isolated Docker browser: navigates, every action across
everyday scenarios container + Chromium fills forms, clicks all 5 layers of data
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ "Book a pet │ ──► │ Container │ ──► │ AI Agent │ ──► │ 5 layers │
│ sitter on │ │ + Chromium │ │ browses the │ │ intercepted │
│ Rover" │ │ + Agent │ │ live site │ │ & recorded │
└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
Live websites · isolated containers · request interception · five-layer recording.
┌─────────────────────────────────────────────────┐
│ Container (Docker / Podman) │
│ │
│ ┌──────────┐ CDP Fetch/Runtime/Page events │
│ │ Chromium ├─────────────────────────────┐ │
│ │ :9222 CDP│ │ │
│ └──────────┘ │ │
│ │ │
│ ┌──────────┐ ┌────────────────▼─┐ │
│ │ Xvfb │◄──ffmpeg──►│ FastAPI Server │ │
│ │ :99 │ x11grab │ :7878 │ │
│ └──────────┘ └──────────────────┘ │
│ │ │
│ ┌───────▼─────────┐ │
│ │ /data │ │
│ │ actions.jsonl │ │
│ │ requests.jsonl │ │
│ │ screenshots/ │ │
│ │ recording.mp4 │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────┘
ClawBench ships three Hugging Face datasets — task definitions plus full execution traces for V1 and V2. All open, downloadable in one command.
| Dataset | What's in it | Get it |
|---|---|---|
| NAIL-Group/ClawBench (mirrored at TIGER-Lab/ClawBench) | Task definitions, rubrics, and metadata for V1 and V2 — what to attempt and how it's judged. | hf download --repo-type dataset NAIL-Group/ClawBench |
| NAIL-Group/ClawBenchV1Trace | One directory per V1 model run: recording.mp4, requests.jsonl, actions.jsonl, agent-messages.jsonl, interception.json, run-meta.json. |
hf download --repo-type dataset NAIL-Group/ClawBenchV1Trace |
| TIGER-Lab/ClawBenchV2Trace | Same 5-layer bundle for V2 runs. Rolling — new models added as they're evaluated. | hf download --repo-type dataset TIGER-Lab/ClawBenchV2Trace |
Build with the traces: See the Trace Cookbook for download recipes, data schemas, and research directions.
The trace datasets are large; use
hf download --include "<pattern>"to pull a single model or a single task.
Live leaderboard:
claw-bench.com/leaderboard(V2 default, two-stage scoring — interception + LLM judge). Full scoring formula ineval/scoring.md. Add your run: PR toleaderboard/results.csv.