axion1_0

1. Highlights

• Three-stage training. Two stages of supervised fine-tuning (SFT) followed by reinforcement learning (RL), without dedicated mid-training, DPO, or on-policy distillation.
• Scientific reasoning and tool use. Axion 1.0-122B scores 59.0 on full HLE with restricted website access, 87.0 on FrontierScience Olympiad, and 66.7 on FrontierScience Research. Axion 1.0-35B scores 62.2 on SEAL-0.
• Verifiable training data. Reasoning problems are strengthened using feedback from the base model. Tasks requiring tools are constructed from public data sources with programmatically determined reference answers.
• Open training code. The repository includes data generation and filtering, prompts, two-stage SFT, asynchronous RL, and benchmark preparation and grading.

2. Results

Selected results from Table 1 of the technical report:

BMB denotes the human-difficult subset of BioMysteryBench; FS denotes FrontierScience. * marks baselines evaluated by the Axion team; other baseline scores are from official reports. — denotes an unreported result.

2.1 Models

Both models in the paper use SFT and RL. The paper also reports 4B and 9B variants trained with SFT only.

We encourage users to try the Axion 1.0-35B community reproduction, announced in issue #1. This is a community release, not an official Axion checkpoint. See the model card for reproduction results and inference instructions.

Training data is available on Hugging Face.

3. Training Recipe

3.1 Stage I: Self-Contained Reasoning

Stage I uses 10,819 trajectories of scientific and mathematical reasoning. Public seed problems are rewritten into harder questions. Problems solved by the base model are further strengthened through composition, parameterization, range shifts, or case analysis, with reference answers checked after each rewrite. Solvers may use Python, but not search or webpage retrieval.

3.2 Stage II: Tool-Dependent Reasoning

Stage II uses 7,544 trajectories, approximately 312M tokens. Programs filter records, select a unique entity, and compute a reference answer before an LLM formulates the question. Solvers must find the required evidence through tools; construction data and answers are withheld during solving. Questions solvable without retrieval are removed.

Generators cover individual entities, ranked lists, and chains across databases, including the NASA Exoplanet Archive, OurAirports, OpenAlex, UniProt, and Ensembl.

3.3 Reinforcement Learning

Starting from the Stage II checkpoint, asynchronous GRPO trains on agent interactions with search, webpage visits, scholarly retrieval, and Python. Correct final answers receive full reward; verified intermediate progress can receive partial credit. Context condensation divides long rollouts into sessions for training.

3.4 Training Infrastructure

The paper's SFT runs use a 128K context window with FSDP, expert parallelism, and context parallelism. RL separates rollout workers, trainers, and auxiliary services into distinct GPU pools. Token-In-Token-Out (TITO) preserves token IDs, and Rollout Routing Replay (R3) aligns MoE expert choices between rollout and training.

Two-stage SFT costs reported in Section 5.1:

These costs are reported in the paper. The released 122B SFT recipe uses 32 GPUs, a 128K context window, and a global batch of 8. Its saved training arguments record micro-batch size 1; Section 5.1 lists 2. The released SFT recipe preserves the saved training arguments.

4. Getting Started

Clone the repository and follow the training guide for environment setup, two-stage SFT and checkpoint export. SFT uses Swift with Megatron-FSDP; RL uses verl. They implement successive stages of the same training pipeline.

git clone https://github.com/bytedance/axion1_0.git
cd axion1_0

axion/
  training/
    sft/         Two-stage SFT, runtime patches and checkpoint export
    rl/          Agent rollouts, rewards, tools and RL configuration
  datagen/       Stage I and Stage II data construction
  evaluation/    Benchmark preparation and grading
third_party/
  swift/         SFT backend
  verl/          RL backend
tests/           Regression and packaging checks

The 122B pipeline consists of two-stage SFT followed by asynchronous GRPO. The training guides cover single-host and multi-host layouts, required services, checkpoint export, and runtime requirements.

For data construction and evaluation, use a separate Python 3.11+ environment:

python -m pip install -e ".[data,evaluation]"

4.1 Inference

Export a native SFT checkpoint to Hugging Face format before serving it. The inference guide provides a vLLM launch command and an OpenAI-compatible request example. Agent evaluation also requires a tool-execution harness.

4.2 Documentation

Evaluation scripts prepare datasets and grade rollouts collected with an external agent harness. The evaluation guide documents the required format.

5. License

Apache 2.0.

6. Citation

@techreport{jia2026axion,
  title  = {Axion 1.0: Open Agent Foundation Models, Simplified},
  author = {Jia, Yuqi and Tian, Yunjie and Zheng, Jiayu and Liang, Yijun
            and Sun, Jingchen and Zhao, Mengyu and Fu, Di},
  year   = {2026},
  url    = {https://axion1-0.github.io/paper/axion.pdf}
}