Tradeoff energy bost / performance
Thanks for this great tool that enables us to compute the energy cost of our LLM, very useful.
But the leaderboard shows ranks only according to the energy cost, while the most important is obviously the tradeoff between energy cost and performance. Isn't there a way to measure this tradeoff? Otherwise, we should all use distilgpt2 ! ;-)
Thank you,
Christophe
Hi Christophe! Of course the tradeoff between energy consumption and performance is important, but every user may assess it differently as it depends on their deployment environment and their goals. This leaderboard enables users to know more about the energy consumption of AI models, while the Open LLM Leaderboard and the LLM Perf Leaderboard enables them to get more information about their performance. The combination of these 3 leaderboards helps to make an informed decision taking into account accuracy, speed and energy consumption.
If you believe there is an objective way of measuring this tradeoff, do not hesitate to propose it here or to open a PR :)