Awesome RL for Agents

A curated list of reinforcement learning (RL) for agents.

This list collects papers, tools, and demos that demonstrate how reinforcement learning can be applied to train or tune LLM/MLLM agents, with a focus on research-driven, computer-using, and tool-integrated agent behaviors. It is not associated with any survey or review — just a personal, living collection of resources on RL for agents. I’ll keep updating it as long as I’m still working in this area.

Table of Contents

• 📚 Papers & Research
• 🕹️ Benchmarks
• 🧪 Demos & Projects
• 🧰 Toolkits & Frameworks
• 📄 Tutorials & Blog Posts
• 🔗 Related Awesome Lists
• 🤝 Contributing

📚 Papers & Research

Survey & Review

• The Landscape of Agentic Reinforcement Learning for LLMs: A Survey [Preprint'25] [AwesomeList]
• A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges [Preprint'25] [AwesomeList]
• Deep Research Agents: A Systematic Examination And Roadmap [Preprint'25] [AwesomeList]

RL for Computer-using Agents

• TMax: A Simple Recipe for Terminal Agents [Preprint'26] [Code] [Blog]
• UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning [Preprint'25] [Code]
• OPENCUA: OpenFoundations for Computer-Use Agents [Preprint'25] [Code]
• ARPO: End-to-End Policy Optimization for GUI Agents with Experience Replay [Preprint'25] [Code]
• InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners [Preprint'25] [Code]
• Cracking the Code of Action: a Generative Approach to Affordances for Reinforcement Learning [Preprint'25]
• UI-R1: Enhancing Action Prediction of GUI Agents by Reinforcement Learning [Preprint'25] [Code]
• Digi-Q: Learning Q-Value Functions for Training Device-Control Agents [Preprint'25] [Code]
• AutoWebGLM: A Large Language Model-based Web Navigating Agent [KDD'24] [Preprint'24] [Code]

RL for Research Agents

• REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agents [Preprint'26] [Code]
• ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking [Preprint'26]
• IterResearch: Rethinking Long-Horizon Agents via Markovian State Reconstruction [Preprint'25]
• Tree Search for LLM Agent Reinforcement Learning [Preprint'25]
• Tongyi DeepResearch: A New Era of Open-Source AI Researchers [Blog] [Code]
• SSRL: Self-Search Reinforcement Learning [Preprint'25] [Code]
• Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RL [Preprint'25] [Code]
• MiroMind Open Deep Research [Blog] [Code]
• ARPO: Agentic Reinforced Policy Optimization [Preprint'25] [Code]
• Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training [Preprint'25] [Code]
• WebShaper: Towards Autonomous Information Seeking Agency [Preprint'25] [Code]
• WebSailor: Navigating Super-human Reasoning for Web Agent [Preprint'25] [Code]
• Kimi-Researcher: End-to-End RL Training for Emerging Agentic Capabilities [Blog]
• R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning [Preprint'25] [Code]
• R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning [Preprint'25] [Code]
• ZeroSearch: Incentivize the Search Capability of LLMs without Searching [Preprint'25] [Code]
• DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments [EMNLP'25] [Code]
• ReCall: Learning to Reason with Tool Call for LLMs via Reinforcement Learning [Preprint'25] [Code]
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning [COLM'25] [Code]
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning [Preprint'25] [Code]
• Agentic Reasoning: Reasoning LLMs with Tools for the Deep Research [Preprint'25] [Code]

RL for Tool-using Problem Solver

• Incentivizing Agentic Reasoning in LLM Judges via Tool-Integrated Reinforcement Learning [Preprint'25]
• VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use [Preprint'25] [Code]
• VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection [Preprint'25] [Code]
• OTC: Optimal Tool Calls via Reinforcement Learning [Preprint'25]
• ToolRL: Reward is All Tool Learning Needs [Preprint'25] [Code]
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs [Preprint'25]
• Agent models: Internalizing Chain-of-Action Generation into Reasoning models [Preprint'25] [Code]
• TORL: Scaling Tool-Integrated RL [Preprint'25] [Code]

Self-Playing Agent with RL

• Toward Training Superintelligent Software Agents through Self-Play SWE-RL [Preprint'25]
• Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning [Preprint'25] [Code]
• Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning [Preprint'25] [Code]
• Search Self-play: Pushing the Frontier of Agent Capability without Supervision [ICLR'26] [Code]

RL for Agent Memory

• MemAgent: Reshaping Long-Context LLM with Multi-Conv RL based Memory Agent [Preprint'25] [Code]
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents [Preprint'25]

RL for Multi-Modal Agent (Thinking w Image / MMSearch)

• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception [Preprint'26] [Code]
• DeepEyesV2: Toward Agentic Multimodal Model [ICLR'26] [Code]
• MMSearch-R1: Incentivizing LMMs to Search [Preprint'25] [Code]
• DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning [ICLR'26] [Code]

RL with Agent Skills

• SkillRL: Evolving Agents via Recursive Skill-Augmente