Leaderboard not updating?

#863
by Pretergeek - opened

Hello,

I hope I am not missing something, because I haven't seen anyone else mentioning this even though it seems to be going on for a few days. But it looks like the leaderboard (and the pending, running and results queues) are not updating even though the requests and results datasets have been updated more recently.

Open LLM Leaderboard org

Hi!

The leaderboard sometimes takes a bit of time to update, so it's usually better to wait for about a day.

Thank you,

Apologies if I am being a bit impatient. It is just that a model I submitted 5 days ago and whose results are already in the results dataset still shows as pending in the leaderboard's space, while other 4 models that I submitted 2 days ago don't even show as pending (although 2 of them are now running according to the requests dataset). I thought that maybe there was some problem that went unreported during the weekend.

Open LLM Leaderboard org

It's possible! Thanks for the details, I'm going to check our different spaces

Open LLM Leaderboard org

One of our parsers indeed went to sleep! I restarted it (and our other spaces), tell me if that works for you!

Indeed, it seems that two of the models that were not even showing as pending now show as running and I received the PR to add the evaluations for the earlier model that was already in the results dataset but not showing in the leaderboard.

Thank you.

clefourrier changed discussion status to closed

Sign up or log in to comment