ehartford/dolphin-2.1-mistral-7b disappeared from the leaderboard

#328
by monology - opened

Where did it go?

I just noticed ehartford/dolphin-2.1-mistral-7b has disappeared from the leaderboard.

Was this some accident?

Open LLM Leaderboard org

Hi!
If you need the results urgently, they are still available here, with the full details here.
I'm therefore unsure as to why it does not appear in the leaderboard, but we updated the caching system recently, maybe there's a bug somewhere. We'll look it up!

Thanks for the links!

Not only this, it also appears the original, pretrained, Mistral-7B has dissapeared, all three submissions of it. Also, metrics evolution over time says there's a model with ~76 ARC which can't be found on the leaderboard.

image.png
???

It's still not there 😔

Open LLM Leaderboard org
edited Oct 19, 2023

Hi @ehartford ! We're investigating as fast as we can, but we are also preparing a big update for the end of the month which is taking most of our time. Thank you for your patience!

Thank you for investigating. Super appreciate it!

Open LLM Leaderboard org

Hi @ehartford thanks for your patience ! It is now fixed :)

SaylorTwift changed discussion status to closed

Thanks You are the best!

again dolphin-2.1-mistral-7b has disappeared

Open LLM Leaderboard org

Hi! Have you checked the request file? :)

The request files for both float16 and bfloat16 precision have status=FINISHED and haven't changed in more than 2 weeks according to the Git history, yet they've recently disappeared.
Could this be an issue on your end, perhaps some lingering problem after the big update?

EDIT: they're fine now, must have just been a glitch at time of posting

monology changed discussion status to open
Open LLM Leaderboard org

Are these the two models you are talking about? If yes they are displayed on the leaderboard

image.png

clefourrier changed discussion status to closed

Sign up or log in to comment