Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery
Quick Start | Submit Tasks | How It Works | Domains | Leaderboard | Add Your Agent
ResearchClawBench is a benchmark that measures whether AI coding agents can independently conduct scientific research — from reading raw data to producing publication-quality reports — and then rigorously evaluates the results against real human-authored papers.
Unlike benchmarks that test coding ability or factual recall, ResearchClawBench asks: given a curated scientific workspace and the same research goal, can an AI agent arrive at the same (or better) scientific conclusions?
Overview
✨ Highlights
🎬 Demo
https://github.com/user-attachments/assets/94829265-80a8-4d61-a744-3800603de6d9
💡 Why ResearchClawBench?
Most AI benchmarks evaluate what models know. We evaluate what agents can do.
- Real science, not toy problems. 40 tasks sourced from published papers across 10 disciplines, each with curated experimental datasets.
- Two-stage pipeline. Autonomous research first, rigorous evaluation second — just like peer review.
- Fine-grained, multimodal scoring. A weighted rubric (checklist) with text and image criteria, judged by an LLM acting as a strict peer reviewer.
- Agent-agnostic. Ships with built-in support for Claude Code, Codex CLI, ARIS Codex, OpenClaw, Nanobot, EvoScientist, ResearchClaw, and a lightweight ResearchHarness baseline. Bring your own agent in one line.
- From Re-Discovery to New-Discovery. Scoring above 50 means matching the original paper; above 70 means surpassing it. The frontier is wide open.
📢 News
- 2026-06-09 📄 ResearchClawBench paper preprint is available on arXiv.
- 2026-06-09 📊 Evaluated MiniMax-M3 as an additional standalone LLM with ResearchHarness. Results are available on the Leaderboard.
- 2026-06-05 🧪 Added
rcb-eval, a YAML-configured command-line evaluation workflow powered by ResearchHarness, supporting concurrent runs, repeated trials, automatic scoring, and Markdown evaluation reports with per-run and per-task statistics. - 2026-06-03 🧬 Added leaderboard results for EvoScientist v0.1.1 while retaining EvoScientist v0.0.4 as a separate versioned baseline. Results are available on the Leaderboard.
- 2026-06-02 📊 Evaluated Claude-Opus-4.8 as an additional standalone LLM with ResearchHarness. Results are available on the Leaderboard.
- 2026-05-28 📊 Evaluated two additional standalone LLMs with ResearchHarness: Gemini-3.5-Flash and Qwen3.7-Max. Results are available on the Leaderboard.
- 2026-05-21 📊 Evaluated four additional standalone LLMs with ResearchHarness: Grok-4.3, Kimi-K2.6, MiMo-V2.5, and DeepSeek-V4-Pro. Results are available on the Leaderboard.
- 2026-04-30 📊 Evaluated standalone LLMs with ResearchHarness: Claude-Opus-4.7, Claude-Opus-4.6, GLM-5.1, Qwen3.6-Plus, Qwen3.5-397B-A17B, GPT-5.5, GPT-5.4, MiMo-V2-Pro, Kimi-K2.5, Grok-4.1, and Gemini-3.1-Pro. Results are available on the Leaderboard.
- 2026-04-13 🧭 Added built-in ARIS Codex UI support and documentation. Imported benchmark runs are supported in the leaderboard and run browser; one-click launch is not yet supported in the public preset.
- 2026-04-10 🔬 Added built-in ResearchClaw support — an intelligent agent-powered research assistant with built-in skills for paper search, literature review, and data analysis.
- 2026-04-07 🧪 Added built-in ResearchHarness support as a lightweight baseline agent for testing different LLMs under the same ResearchClawBench workflow.
- 2026-03-30 🧬 Added built-in EvoScientist support and clarified multimodal judge prompting so the first attached image is explicitly treated as the ground-truth figure.
- 2026-03-27 🤗 Released a Hugging Face dataset mirror at InternScience/ResearchClawBench, including 10 additional tasks from ResearchClawBench-Self and a task downloader script.
- 2026-03-27 📨 Opened the ResearchClawBench submission Space for community task uploads. New tasks are validated there and reviewed through Hugging Face dataset PRs instead of being added to this GitHub repository.
- 2026-03-20 🐈 Added Nanobot as a new agent — ultra-lightweight OpenClaw alternative with reliable multi-step tool execution. Agent config moved to
agents.jsonfor easy customization. - 2026-03-19 🚀 Initial release with Claude Code, Codex CLI, and OpenClaw support. 40 tasks across 10 scientific domains.
Understanding The Benchmark
🏗️ Data Construction
Every task in ResearchClawBench is built through a rigorous, expert-driven pipeline to ensure scientific validity and reproducibility:
flowchart TD
A["📄 High-Quality Paper Collection\n(Target Paper)"] --> B["🧑🔬 Human Expert Extraction\n(Core Task Instructions)"]
B --> C["📋 Evaluation Rubric (Checklist)\n(Criteria + Keywords + Weights)"]
B --> D["📂 Data & Related Work Collection\n(Datasets + Reference Papers)"]
C --> E["✅ Human Reproduction & Validation\n(Verify rubric (checklist) is reproducible)"]
D --> E
style A fill:#e0f2fe,stroke:#0284c7,stroke-width:2px
style B fill:#fef3c7,stroke:#f59e0b,stroke-width:2px
style C fill:#fce7f3,stroke:#ec4899,stroke-width:2px
style D fill:#f0fdf4,stroke:#22c55e,stroke-width:2px
style E fill:#f5f3ff,stroke:#8b5cf6,stroke-width:2px
-
High-Quality Paper Collection — Domain experts select recent, high-impact publications with clear methodology and reproducible results across 10 scientific disciplines.
-
Expert Task Extraction — Human experts read each paper and distill the core research task into structured instructions, identifying the key scientific question, input data, and expected outputs.
-
Rubric (Checklist) Design — Experts create a fine-grained evaluation rubric (checklist) with weighted criteria (text and image items), each with specific technical keywords that a judge must verify.
-
Data & Related Work Collection — The datasets and related reference materials are curated to form a research workspace for the task.
-
Human Reproduction & Validation — Human researchers independently reproduce the paper's results from the provided workspace and instructions, verifying that every rubric (checklist) item is achievable. This ensures the benchmark is fair and the rubric (checklist) is grounded in r