ResearchClawBench

Name: ResearchClawBench
Author: InternScience

Verified

🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

195stars

17forks

Jupyter Notebook

Installation

# Add to your Claude Code skills
git clone https://github.com/InternScience/ResearchClawBench

Getting Started

Guides for using ai agents skills like ResearchClawBench.

Caveman: Cut Claude Token Use by 65%
How agent-side prompt compression works, when to use it, and when not to.
What is an AI Skills Marketplace?
Definitions, how marketplaces work, and how to choose between them in 2026.
Getting Started with AI Skills

Security ReportVerified

Last scanned: 5/30/2026

{
  "issues": [],
  "status": "PASSED",
  "scannedAt": "2026-05-30T16:25:40.966Z",
  "npmAuditRan": true,
  "pipAuditRan": true
}

README.md

Frequently Asked Questions

What is ResearchClawBench?

ResearchClawBench is an open-source ai agents skill for AI coding assistants such as Claude Code, Codex CLI, and ChatGPT, built by InternScience. 🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery. It has 195 GitHub stars.

Is ResearchClawBench safe to use?

Yes. ResearchClawBench passed SkillsLLM's automated security scan — a dependency vulnerability audit plus prompt-injection heuristics — with no high-severity issues. You can read the full report in the Security Report section on this page.

How do I install ResearchClawBench?

Clone the repository with "git clone https://github.com/InternScience/ResearchClawBench" and add it to your Claude Code skills directory (see the Installation section above).

What programming language is ResearchClawBench written in?

ResearchClawBench is primarily written in Jupyter Notebook. It is open-source under InternScience on GitHub, so you can review or fork the full source.

Are there alternatives to ResearchClawBench?

Yes. SkillsLLM lists many other AI Agents skills you can browse and compare side by side. Open the AI Agents category from the badge at the top of this page, or use the Related Skills and comparison links further down to weigh ResearchClawBench against similar tools.

Agentic AI for Beginners

Build your first AI agent from scratch - tool use, ReAct pattern, memory, deployment

41 minBeginner

Comments (0)

to leave a comment.

No comments yet. Be the first to share your thoughts!

Related Skills

superpowers

by obra

An agentic skills framework & software development methodology that works.

234,966

MCP-Checklists agentlogs

Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

ResearchClawBench is a benchmark that measures whether AI coding agents can independently conduct scientific research — from reading raw data to producing publication-quality reports — and then rigorously evaluates the results against real human-authored papers.

Unlike benchmarks that test coding ability or factual recall, ResearchClawBench asks: given a curated scientific workspace and the same research goal, can an AI agent arrive at the same (or better) scientific conclusions?

Overview

✨ Highlights

🎬 Demo

https://github.com/user-attachments/assets/94829265-80a8-4d61-a744-3800603de6d9

💡 Why ResearchClawBench?

Most AI benchmarks evaluate what models know. We evaluate what agents can do.

Real science, not toy problems. 40 tasks sourced from published papers across 10 disciplines, each with curated experimental datasets.
Two-stage pipeline. Autonomous research first, rigorous evaluation second — just like peer review.
Fine-grained, multimodal scoring. A weighted rubric (checklist) with text and image criteria, judged by an LLM acting as a strict peer reviewer.
Agent-agnostic. Ships with built-in support for Claude Code, Codex CLI, ARIS Codex, OpenClaw, Nanobot, EvoScientist, ResearchClaw, and a lightweight ResearchHarness baseline. Bring your own agent in one line.
From Re-Discovery to New-Discovery. Scoring above 50 means matching the original paper; above 70 means surpassing it. The frontier is wide open.

📢 News

2026-07-09 📊 Evaluated Open Science with Claude-Opus-4.8 as an additional autonomous research agent. Results are available on the Leaderboard.
2026-07-08 📊 Evaluated Hy3-Preview, a preview model rather than a final release, as an additional standalone LLM with ResearchHarness. Results are available on the Leaderboard.
2026-07-01 📈 Added Pass@5 leaderboard results, including stability-oriented statistics across repeated runs. Results are available on the Leaderboard.
2026-06-24 📊 Evaluated GLM-5.2 as an additional standalone LLM with ResearchHarness. Results are available on the Leaderboard.
2026-06-09 📄 ResearchClawBench paper preprint is available on arXiv.
2026-06-09 📊 Evaluated MiniMax-M3 as an additional standalone LLM with ResearchHarness. Results are available on the Leaderboard.

2026-06-05 🧪 Added rcb-eval, a YAML-configured command-line evaluation workflow powered by ResearchHarness, supporting concurrent runs, repeated trials, automatic scoring, and Markdown evaluation reports with per-run and per-task statistics.
2026-06-03 🧬 Added leaderboard results for EvoScientist v0.1.1 while retaining EvoScientist v0.0.4 as a separate versioned baseline. Results are available on the Leaderboard.
2026-06-02 📊 Evaluated Claude-Opus-4.8 as an additional standalone LLM with ResearchHarness. Results are available on the Leaderboard.
2026-05-28 📊 Evaluated two additional standalone LLMs with ResearchHarness: Gemini-3.5-Flash and Qwen3.7-Max. Results are available on the Leaderboard.
2026-05-21 📊 Evaluated four additional standalone LLMs with ResearchHarness: Grok-4.3, Kimi-K2.6, MiMo-V2.5, and DeepSeek-V4-Pro. Results are available on the Leaderboard.
2026-04-30 📊 Evaluated standalone LLMs with ResearchHarness: Claude-Opus-4.7, Claude-Opus-4.6, GLM-5.1, Qwen3.6-Plus, Qwen3.5-397B-A17B, GPT-5.5, GPT-5.4, MiMo-V2-Pro, Kimi-K2.5, Grok-4.1, and Gemini-3.1-Pro. Results are available on the Leaderboard.
2026-04-13 🧭 Added built-in ARIS Codex UI support and documentation. Imported benchmark runs are supported in the leaderboard and run browser; one-click launch is not yet supported in the public preset.
2026-04-10 🔬 Added built-in ResearchClaw support — an intelligent agent-powered research assistant with built-in skills for paper search, literature review, and data analysis.
2026-04-07 🧪 Added built-in ResearchHarness support as a lightweight baseline agent for testing different LLMs under the same ResearchClawBench workflow.
2026-03-30 🧬 Added built-in EvoScientist support and clarified multimodal judge prompting so the first attached image is explicitly treated as the ground-truth figure.
2026-03-27 🤗 Released a Hugging Face dataset mirror at InternScience/ResearchClawBench, including 10 additional tasks from ResearchClawBench-Self and a task downloader script.
2026-03-27 📨 Opened the ResearchClawBench submission Space for community task uploads. New tasks are validated there and reviewed through Hugging Face dataset PRs instead of being added to this GitHub repository.
2026-03-20 🐈 Added Nanobot as a new agent — ultra-lightweight OpenClaw alternative with reliable multi-step tool execution. Agent config moved to agents.json for easy customization.
2026-03-19 🚀 Initial release with Claude Code, Codex CLI, and OpenClaw support. 40 tasks across 10 scientific domains.

Understanding The Benchmark

🏗️ Data Construction

Every task in ResearchClawBench is built through a rigorous, expert-driven pipeline to ensure scientific validity and reproducibility:

flowchart TD
    A["📄 High-Quality Paper Collection\n(Target Paper)"] --> B["🧑‍🔬 Human Expert Extraction\n(Core Task Instructions)"]
    B --> C["📋 Evaluation Rubric (Checklist)\n(Criteria + Keywords + Weights)"]
    B --> D["📂 Data & Related Work Collection\n(Datasets + Reference Papers)"]
    C --> E["✅ Human Reproduction & Validation\n(Verify rubric (checklist) is reproducible)"]
    D --> E

    style A fill:#e0f2fe,stroke:#0284c7,stroke-width:2px
    style B fill:#fef3c7,stroke:#f59e0b,stroke-width:2px