Running
63
๐
Open LLM Leaderboard Model Comparator
Compare Open LLM Leaderboard results
Albert Villanova del Moral
albertvillanova
AI & ML interests
ML Engineer @ Hugging Face: Evaluations (Science)
Recent Activity
updated
a Space
4 days ago
open-llm-leaderboard/comparator
posted
an
update
5 days ago
๐จ How green is your model? ๐ฑ Introducing a new feature in the Comparator tool: Environmental Impact for responsible #LLM research!
๐ https://huggingface.co/spaces/open-llm-leaderboard/comparator
Now, you can not only compare models by performance, but also by their environmental footprint!
๐ The Comparator calculates COโ emissions during evaluation and shows key model characteristics: evaluation score, number of parameters, architecture, precision, type... ๐ ๏ธ
Make informed decisions about your model's impact on the planet and join the movement towards greener AI!
updated
a dataset
6 days ago
albertvillanova/tmp-state-on-load-ds
Organizations
albertvillanova's activity
updated
a
Space
4 days ago
posted
an
update
5 days ago
Post
1090
๐จ How green is your model? ๐ฑ Introducing a new feature in the Comparator tool: Environmental Impact for responsible #LLM research!
๐ open-llm-leaderboard/comparator
Now, you can not only compare models by performance, but also by their environmental footprint!
๐ The Comparator calculates COโ emissions during evaluation and shows key model characteristics: evaluation score, number of parameters, architecture, precision, type... ๐ ๏ธ
Make informed decisions about your model's impact on the planet and join the movement towards greener AI!
๐ open-llm-leaderboard/comparator
Now, you can not only compare models by performance, but also by their environmental footprint!
๐ The Comparator calculates COโ emissions during evaluation and shows key model characteristics: evaluation score, number of parameters, architecture, precision, type... ๐ ๏ธ
Make informed decisions about your model's impact on the planet and join the movement towards greener AI!
updated
a
dataset
6 days ago
updated
a
Space
6 days ago
upvoted
a
collection
15 days ago
upvoted
a
collection
19 days ago
posted
an
update
19 days ago
Post
1238
๐ New feature of the Comparator of the ๐ค Open LLM Leaderboard: now compare models with their base versions & derivatives (finetunes, adapters, etc.). Perfect for tracking how adjustments affect performance & seeing innovations in action. Dive deeper into the leaderboard!
๐ ๏ธ Here's how to use it:
1. Select your model from the leaderboard.
2. Load its model tree.
3. Choose any base & derived models (adapters, finetunes, merges, quantizations) for comparison.
4. Press Load.
See side-by-side performance metrics instantly!
Ready to dive in? ๐ Try the ๐ค Open LLM Leaderboard Comparator now! See how models stack up against their base versions and derivatives to understand fine-tuning and other adjustments. Easier model analysis for better insights! Check it out here: open-llm-leaderboard/comparator ๐
๐ ๏ธ Here's how to use it:
1. Select your model from the leaderboard.
2. Load its model tree.
3. Choose any base & derived models (adapters, finetunes, merges, quantizations) for comparison.
4. Press Load.
See side-by-side performance metrics instantly!
Ready to dive in? ๐ Try the ๐ค Open LLM Leaderboard Comparator now! See how models stack up against their base versions and derivatives to understand fine-tuning and other adjustments. Easier model analysis for better insights! Check it out here: open-llm-leaderboard/comparator ๐
posted
an
update
26 days ago
Post
3094
๐ Exciting update! You can now compare multiple models side-by-side with the Hugging Face Open LLM Comparator! ๐
open-llm-leaderboard/comparator
Dive into multi-model evaluations, pinpoint the best model for your needs, and explore insights across top open LLMs all in one place. Ready to level up your model comparison game?
open-llm-leaderboard/comparator
Dive into multi-model evaluations, pinpoint the best model for your needs, and explore insights across top open LLMs all in one place. Ready to level up your model comparison game?
upvoted
an
article
29 days ago
Article
Let's talk about LLM evaluation
By
โข
โข
134
upvoted
2
collections
30 days ago
updated
a
Space
about 1 month ago
posted
an
update
about 1 month ago
Post
1214
๐จ Instruct-tuning impacts models differently across families! Qwen2.5-72B-Instruct excels on IFEval but struggles with MATH-Hard, while Llama-3.1-70B-Instruct avoids MATH performance loss! Why? Can they follow the format in examples? ๐ Compare models:
open-llm-leaderboard/comparator
updated
a
Space
about 1 month ago
Dataset Viewer issue: TypeError: Couldn't cast array
6
#5 opened 2 months ago
by
albertvillanova
albertvillanova
upvoted
an
article
about 1 month ago
Article
SmolLM - blazingly fast and remarkably powerful
โข
267
posted
an
update
about 1 month ago
Post
1905
Finding the Best SmolLM for Your Project
Need an LLM assistant but unsure which hashtag#smolLM to run locally? With so many models available, how can you decide which one suits your needs best? ๐ค
If the model youโre interested in is evaluated on the Hugging Face Open LLM Leaderboard, thereโs an easy way to compare them: use the model Comparator tool: open-llm-leaderboard/comparator
Letโs walk through an example๐
Letโs compare two solid options:
- Qwen2.5-1.5B-Instruct from Alibaba Cloud Qwen (1.5B params)
- gemma-2-2b-it from Google (2.5B params)
For an assistant, you want a model thatโs great at instruction following. So, how do these two models stack up on the IFEval task?
What about other evaluations?
Both models are close in performance on many other tasks, showing minimal differences. Surprisingly, the 1.5B Qwen model performs just as well as the 2.5B Gemma in many areas, even though it's smaller in size! ๐
This is a great example of how parameter size isnโt everything. With efficient design and training, a smaller model like Qwen2.5-1.5B can match or even surpass larger models in certain tasks.
Looking for other comparisons? Drop your model suggestions below! ๐
Need an LLM assistant but unsure which hashtag#smolLM to run locally? With so many models available, how can you decide which one suits your needs best? ๐ค
If the model youโre interested in is evaluated on the Hugging Face Open LLM Leaderboard, thereโs an easy way to compare them: use the model Comparator tool: open-llm-leaderboard/comparator
Letโs walk through an example๐
Letโs compare two solid options:
- Qwen2.5-1.5B-Instruct from Alibaba Cloud Qwen (1.5B params)
- gemma-2-2b-it from Google (2.5B params)
For an assistant, you want a model thatโs great at instruction following. So, how do these two models stack up on the IFEval task?
What about other evaluations?
Both models are close in performance on many other tasks, showing minimal differences. Surprisingly, the 1.5B Qwen model performs just as well as the 2.5B Gemma in many areas, even though it's smaller in size! ๐
This is a great example of how parameter size isnโt everything. With efficient design and training, a smaller model like Qwen2.5-1.5B can match or even surpass larger models in certain tasks.
Looking for other comparisons? Drop your model suggestions below! ๐
updated
2
Spaces
about 1 month ago
search selection list autocomplete
1
#1 opened about 1 month ago
by
Aurelien-Morgan
Aurelien-Morgan