smolagents-benchmark/answers
Viewer
โข
Updated
โข
2.38k
โข
35
Aymeric Roucher
m-ric
AI & ML interests
Leading Agents at Hugging Face ๐ค
Recent Activity
updated
a dataset
19 minutes ago
smolagents-benchmark/answers
upvoted
an
article
24 minutes ago
Introducing Three New Serverless Inference Providers: Hyperbolic, Nebius AI Studio, and Novita ๐ฅ
updated
a Space
about 7 hours ago
smolagents-benchmark/smolagents_llm_leaderboard
Organizations
m-ric's activity
updated
a
dataset
19 minutes ago
upvoted
an
article
24 minutes ago
Article
Introducing Three New Serverless Inference Providers: Hyperbolic, Nebius AI Studio, and Novita ๐ฅ
โข
26
updated
a
Space
about 7 hours ago
liked
a
dataset
about 8 hours ago
posted
an
update
about 10 hours ago
Post
326
Less is More for Reasoning (LIMO): a 32B model fine-tuned with 817 examples can beat o1-preview on math reasoning! ๐คฏ
Do we really need o1's huge RL procedure to see reasoning emerge? It seems not.
Researchers from Shanghai Jiaotong University just demonstrated that carefully selected examples can boost math performance in large language models using SFT โno huge datasets or RL procedures needed.
Their procedure allows Qwen2.5-32B-Instruct to jump from 6.5% to 57% on AIME and from 59% to 95% on MATH, while using only 1% of the data in previous approaches.
โก The Less-is-More Reasoning Hypothesis:
โฃ Minimal but precise examples that showcase optimal reasoning patterns matter more than sheer quantity
โฃ Pre-training knowledge plus sufficient computational resources at inference levels up math skills
โก๏ธ Core techniques:
โฃ High-quality reasoning chains with self-verification steps
โฃ 817 handpicked problems that encourage deeper reasoning
โฃ Enough inference-time computation to allow extended reasoning
๐ช Efficiency gains:
โฃ Only 817 examples instead of 100k+
โฃ 40.5% absolute improvement across 10 diverse benchmarks, outperforming models trained on 100x more data
This really challenges the notion that SFT leads to memorization rather than generalization! And opens up reasoning to GPU-poor researchers ๐
Read the full paper here ๐ย LIMO: Less is More for Reasoning (2502.03387)
Do we really need o1's huge RL procedure to see reasoning emerge? It seems not.
Researchers from Shanghai Jiaotong University just demonstrated that carefully selected examples can boost math performance in large language models using SFT โno huge datasets or RL procedures needed.
Their procedure allows Qwen2.5-32B-Instruct to jump from 6.5% to 57% on AIME and from 59% to 95% on MATH, while using only 1% of the data in previous approaches.
โก The Less-is-More Reasoning Hypothesis:
โฃ Minimal but precise examples that showcase optimal reasoning patterns matter more than sheer quantity
โฃ Pre-training knowledge plus sufficient computational resources at inference levels up math skills
โก๏ธ Core techniques:
โฃ High-quality reasoning chains with self-verification steps
โฃ 817 handpicked problems that encourage deeper reasoning
โฃ Enough inference-time computation to allow extended reasoning
๐ช Efficiency gains:
โฃ Only 817 examples instead of 100k+
โฃ 40.5% absolute improvement across 10 diverse benchmarks, outperforming models trained on 100x more data
This really challenges the notion that SFT leads to memorization rather than generalization! And opens up reasoning to GPU-poor researchers ๐
Read the full paper here ๐ย LIMO: Less is More for Reasoning (2502.03387)
liked
a
model
about 11 hours ago
added sidebar layout
#19 opened 2 days ago
by
ysharma
liked
a
Space
4 days ago
updated
a
Space
4 days ago
published
a
Space
4 days ago
upvoted
an
article
4 days ago
Article
1 Billion Classifications
โข
37
commented on
Welcome Fireworks.ai on the Hub ๐
4 days ago
Amazing, looking forward to useing it!
upvoted
an
article
4 days ago
Article
Welcome Fireworks.ai on the Hub ๐
โข
45
posted
an
update
4 days ago
Post
2460
๐๐ฟ๐ฒ๐ฎ๐ ๐ณ๐ฒ๐ฎ๐๐๐ฟ๐ฒ ๐ฎ๐น๐ฒ๐ฟ๐: you can now share agents to the Hub! ๐ฅณ๐ฅณ
And any agent pushed to Hub get a cool Space interface to directly chat with it.
This was a real technical challenge: for instance, serializing tools to export them meant that you needed to get all the source code for a tool, verify that it was standalone (not relying on external variables), and gathering all the packages required to make it run.
Go try it out! ๐ https://github.com/huggingface/smolagents
And any agent pushed to Hub get a cool Space interface to directly chat with it.
This was a real technical challenge: for instance, serializing tools to export them meant that you needed to get all the source code for a tool, verify that it was standalone (not relying on external variables), and gathering all the packages required to make it run.
Go try it out! ๐ https://github.com/huggingface/smolagents
- 2 replies
updated
a
Space
4 days ago
Nope doesn' work, error
#4 opened 13 days ago
by
Gamineer
Chat oiseaux
#5 opened 13 days ago
by
Trucgjgf
Error
3
#7 opened 8 days ago
by
deleted
More steps
2
#16 opened 5 days ago
by
Ismael1-2-3
works great george carlin is alive and hosting a new snl this week
#13 opened 6 days ago
by
timtaco