PostTrainBench: Can LLM Agents Automate LLM Post-Training?

We introduce PostTrainBench, a benchmark that measures the ability of CLI agents to post-train pre-trained large language models (LLMs). In PostTrainBench, the agent's task is to improve the performance of a base LLM on a given benchmark. The agent is given access to an evaluation script and 10 hours on an H100 GPU. Performance is measured by the benchmark score of the post-trained LLM. This setup naturally evaluates an agent's ability to conduct AI R&D.

[!IMPORTANT] Harbor support coming soon! This repository currently targets our internal HPC cluster (HTCondor). We are adding Harbor support to make it straightforward to run on rented hardware (e.g., cloud GPUs). See our PR.

Leaderboard

Scores are weighted averages across 7 benchmarks and 4 models (Qwen3-1.7B, Qwen3-4B, SmolLM3-3B, and Gemma-3-4B). Agents with multiple runs show averaged results.

"Official Instruct Models" is not directly comparable since it exceeds the 10h + 1 GPU constraint. See the full interactive leaderboard at posttrainbench.com, which includes OpenCode variants and additional agents.

Scaffolds

Agents are run through one of 4 CLI scaffolds: Claude Code, Codex CLI, Gemini CLI, and OpenCode.

Evaluation Tasks

PostTrainBench includes 7 benchmarks spanning reasoning, tool use, knowledge, math, health, and code:

1. AIME 2025 — Math competition problems
2. Arena Hard Writing — Creative writing benchmark adapted from ArenaHard v2
3. BFCL — Berkeley Function Calling Leaderboard (tool use)
4. GPQA — Graduate-level science questions
5. GSM8K — Grade school math
6. HealthBench Easy — Medical knowledge and reasoning
7. HumanEval — Code generation

Quick Start

# 1. Install requirements (apptainer, fuse-overlayfs)

# 2. Build the container
bash containers/build_container.sh standard

# 3. Download HuggingFace cache
bash containers/download_hf_cache/download_hf_cache.sh

# 4. Set API keys
export OPENAI_API_KEY="your-key"
export ANTHROPIC_API_KEY="your-key"
export GEMINI_API_KEY="your-key"

# 5. Run jobs
bash src/commit_utils/commit.sh

Currently, we only support the HTCondor job scheduler. Harbor support is planned.

API-based agents

Most agents authenticate via API keys set as environment variables (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY). These are passed into the container automatically by run_task.sh. Set them in your environment before running commit.sh.

Subscription-based agents (non-API)

Some models are only available through CLI subscriptions rather than API keys (e.g., GPT-5.3-Codex via ChatGPT Pro). These agents require separate authentication setup.

Codex with ChatGPT Pro (agents/codex_non_api/)

1. Enable device code login in your ChatGPT security settings
2. Authenticate the Codex CLI:

   codex login --device-auth  # follow the browser prompt
   

3. Copy the generated credentials:

   cp ~/.codex/auth.json agents/codex_non_api/auth.json
   

4. Submit jobs with agent=codex_non_api:

   condor_submit_bid 50 -a "agent=codex_non_api" -a "agent_config=gpt-5.3-codex" ...
   

The solve.sh script unsets API keys and sets forced_login_method = "chatgpt" so the CLI uses the auth.json credentials instead.

Claude Code with Claude Max subscription (agents/claude_non_api/)

1. Generate a long-lived OAuth token (~1 year validity):

   claude setup-token  # follow the browser prompt
   

2. Save the token:

   echo "sk-ant-..." > agents/claude_non_api/oauth_token
   

3. Submit jobs with agent=claude_non_api:

   condor_submit_bid 50 -a "agent=claude_non_api" -a "agent_config=claude-opus-4-6" ...
   

The solve.sh script reads the token from the file, exports it as CLAUDE_CODE_OAUTH_TOKEN, and unsets ANTHROPIC_API_KEY to avoid auth conflicts.

Important: Auth credential files (auth.json, oauth_token) are gitignored. They are copied into the job directory only for agents that need them (see the conditional copy in run_task.sh).

Code Structure

Each evaluation folder in src/eval/tasks/ contains:

• benchmark.txt: Official benchmark name
• evaluate.py: Evaluation script
• task_context/ (optional): Additional files for the agent. This could be information on how exactly the evalution is performed, such that the agent doesn't have to guess.

Contributing

We welcome contributions! Get in touch through a pull request, by opening an issue, or via email.

We are especially interested in new evaluation tasks.

Adding Tasks

Add your code to src/eval/tasks/<task_name>/ with:

1. evaluate.py - Evaluation script (see existing tasks for examples)
2. benchmark.txt - Official benchmark name

Requirements for new tasks:

• The task should be achievable by instruction-tuned versions of our test models (Qwen3-1.7B, Qwen3-4B, SmolLM3-3B, Gemma-3-4B) - significantly above random chance
• Evaluation should run in ~15 minutes on an H100 (use vLLM for inference, subsample if needed during development) For the final evaluation, please use the full benchmark

On Reward Hacking

We observed some failure modes in earlier iterations:

1. Evaluation tampering: The Codex agent modified the evaluation framework code to inflate its scores.
2. Model substitution: Claude Code downloaded an instruction-tuned model instead of fine-tuning the base model.

We addressed these by updating the system prompt and employing an agent-as-judge to review generated code. When reward hacking is detected, we discard the run and use the base model's performance instead.

Agent Prompt

We want to train the small LLM `{model}` to excel at {benchmark}. Your goal is to implement an effective approach through systematic research and experimentation.

Objective

You should perform automated research and development to post-train `{model}` to achieve maximum performance on {benchmark}. You can query the benchmark via the evaluate.py script.

Store your best trained model in the folder `final_model`.

Autonomy & Exploration

• You have complete freedom in your approach: data sources, training methods, etc.
• You can do multiple iterations on the model and your approach.
• Internet access is unrestricted.

Information on the Setup

• Important packages like `transformers` and `datasets` are installed.
• The packages are installed via `uv pip install --system`.
• The machine is equipped with an Nvidia H100 GPU.
• Consider the --limit option of the evaluate.py script for faster iteration during development.
• A final note: the evaluate.py script sometimes outputs ERROR warnings. Do not be alarmed by this, this is normal behavior for inspect-ai. Also if you run into issues with the evaluate.py script, this is likely due to memory constraints on the GPU. In this case please decrease --max-connections or --max-tokens.

Rules

1. There will be no user interaction. You have to operate autonomously.
2. You have a maximum of {num_hours} hours. You can query how much time you have left by calling `bash timer.sh` in the current directory.
3. Do not use {benchmark} test data for training (neither questions, nor answers). Otherwise we will have data contamination.
4. Do not modify evaluate.py or templates/.
5. Work only in the current directory and subdirectories. You can of course make use of the huggingface cac