awesome-foundation-model-leaderboards

<div align="center"> <img src="https://github.com/user-attachments/assets/1165c100-971b-4336-bd75-f5664311643c" alt="lbops" width="360"/> <h1>Awesome AI Leaderboard</h1> <a href="https://awesome.re"> <img src="https://awesome.re/badge.svg" height="20"/> </a> <a href="https://github.com/SAILResearch/awesome-foundation-model-leaderboards/fork"> <img src="https://img.shields.io/badge/PRs-Welcome-red" height="20"/> </a> <a href="https://arxiv.org/pdf/2407.04065.pdf"> <img src="https://img.shields.io/badge/Arxiv-2407.04065-red" height="20"/> </a> </div>

Awesome AI Leaderboard is a curated list of awesome AI leaderboards, along with various development tools and evaluation organizations according to our recent survey:

<p align="center"><strong>On the Workflows and Smells of Leaderboard Operations (LBOps):<br>An Exploratory Study of Foundation Model Leaderboards</strong></p> <p align="center"><a href="https://github.com/zhimin-z">Zhimin (Jimmy) Zhao</a>, <a href="https://abdulali.github.io">Abdul Ali Bangash</a>, <a href="https://www.filipecogo.pro">Filipe Roseiro Côgo</a>, <a href="https://mcis.cs.queensu.ca/bram.html">Bram Adams</a>, <a href="https://research.cs.queensu.ca/home/ahmed">Ahmed E. Hassan</a></p> <p align="center"><a href="https://sail.cs.queensu.ca">Software Analysis and Intelligence Lab (SAIL)</a></p>

If you find this repository useful, please consider giving us a star :star: and citation:

@article{zhao2025workflows,
  title={On the Workflows and Smells of Leaderboard Operations (LBOps): An Exploratory Study of Foundation Model Leaderboards},
  author={Zhao, Zhimin and Bangash, Abdul Ali and C{\^o}go, Filipe Roseiro and Adams, Bram and Hassan, Ahmed E},
  journal={IEEE Transactions on Software Engineering},
  year={2025},
  publisher={IEEE}
}

Additionally, we provide a search toolkit that helps you quickly navigate through the leaderboards.