Software Engineering Arena

community

AI & ML interests

Model Evaluation

Recent Activity

zhiminy  updated a dataset 9 days ago
SE-Arena/conversations
zhiminy  updated a dataset 9 days ago
SE-Arena/votes
View all activity

SE-Arena's activity

zhiminy 
posted an update 3 months ago
zhiminy 
posted an update 8 months ago
zhiminy 
posted an update 8 months ago
view post
Post
1998
Hey everyone!

Our team just dropped something cool! 🎉 We've published a new paper on arxiv diving into the foundation model leaderboards across different platforms. We've analyzed the content, operational workflows, and common issues of these leaderboards. From this, we came up with two new concepts: Leaderboard Operations (LBOps) and leaderboard smells.

We also put together an awesome list with nearly 300 of the latest leaderboards, development tools, and publishing organizations. You can check it out here: https://github.com/SAILResearch/awesome-foundation-model-leaderboards

If you find it useful or interesting, give us a follow or drop a comment. We'd love to hear your thoughts and get your support! ✨

Link to the paper: https://arxiv.org/abs/2407.04065