Llama-based model evaluation failure again

#156
by deleted - opened
deleted

Hello @clefourrier @SaylorTwift , just wondering why my model resulted in failure (according to the requests log)? I remember when GodziLLa-30B (which I'm using now as its base model), was evaluated 2 weeks ago, it also encountered an issue being evaluated, and it turned out to affect a lot of the llama-based models. I don't know if the same thing is happening here.

Can you please help me out? It would be helpful if I could know the error that your evaluation code is returning so that I can fix it on my end. Thank you very much.

Open LLM Leaderboard org

Hi @jaspercatapang ,
Just checked the log and it seems it's entirely on our end. We'll relaunch your model asap.

@clefourrier Same with our Llama-2-70b-instruct-v2. Is there anything we need to fix? Thanks :)

Open LLM Leaderboard org

Same thing - tbh I think we used a slightly unstable version of the backend yesterday, we'll fix it asap 😅
cc: @SaylorTwift fyi

@clefourrier I appreciate your commitment. Cheers!

deleted changed discussion status to closed

Sign up or log in to comment