CohereForAI/c4ai-command-a-03-2025
Text Generation
ā¢
Updated
ā¢
162
Carlo Moro
cnmoro
AI & ML interests
I like small & fast models
Recent Activity
liked
a model
about 13 hours ago
CohereForAI/c4ai-command-a-03-2025
upvoted
a
collection
1 day ago
Gemma 3 Release
liked
a model
1 day ago
google/gemma-3-1b-it
Organizations
cnmoro's activity
liked
a
model
about 13 hours ago
upvoted
a
collection
1 day ago
liked
a
model
1 day ago
liked
a
model
2 days ago
updated
a
model
2 days ago
published
a
model
3 days ago
updated
a
model
3 days ago
updated
a
dataset
3 days ago
reacted to
tomaarsen's
post with ā¤ļø
3 days ago
Post
6201
An assembly of 18 European companies, labs, and universities have banded together to launch šŖšŗ EuroBERT! It's a state-of-the-art multilingual encoder for 15 European languages, designed to be finetuned for retrieval, classification, etc.
šŖšŗ 15 Languages: English, French, German, Spanish, Chinese, Italian, Russian, Polish, Portuguese, Japanese, Vietnamese, Dutch, Arabic, Turkish, Hindi
3ļøā£ 3 model sizes: 210M, 610M, and 2.1B parameters - very very useful sizes in my opinion
ā”ļø Sequence length of 8192 tokens! Nice to see these higher sequence lengths for encoders becoming more common.
āļø Architecture based on Llama, but with bi-directional (non-causal) attention to turn it into an encoder. Flash Attention 2 is supported.
š„ A new Pareto frontier (stronger *and* smaller) for multilingual encoder models
š Evaluated against mDeBERTa, mGTE, XLM-RoBERTa for Retrieval, Classification, and Regression (after finetuning for each task separately): EuroBERT punches way above its weight.
š Detailed paper with all details, incl. data: FineWeb for English and CulturaX for multilingual data, The Stack v2 and Proof-Pile-2 for code.
Check out the release blogpost here: https://huggingface.co/blog/EuroBERT/release
* EuroBERT/EuroBERT-210m
* EuroBERT/EuroBERT-610m
* EuroBERT/EuroBERT-2.1B
The next step is for researchers to build upon the 3 EuroBERT base models and publish strong retrieval, zero-shot classification, etc. models for all to use. I'm very much looking forward to it!
šŖšŗ 15 Languages: English, French, German, Spanish, Chinese, Italian, Russian, Polish, Portuguese, Japanese, Vietnamese, Dutch, Arabic, Turkish, Hindi
3ļøā£ 3 model sizes: 210M, 610M, and 2.1B parameters - very very useful sizes in my opinion
ā”ļø Sequence length of 8192 tokens! Nice to see these higher sequence lengths for encoders becoming more common.
āļø Architecture based on Llama, but with bi-directional (non-causal) attention to turn it into an encoder. Flash Attention 2 is supported.
š„ A new Pareto frontier (stronger *and* smaller) for multilingual encoder models
š Evaluated against mDeBERTa, mGTE, XLM-RoBERTa for Retrieval, Classification, and Regression (after finetuning for each task separately): EuroBERT punches way above its weight.
š Detailed paper with all details, incl. data: FineWeb for English and CulturaX for multilingual data, The Stack v2 and Proof-Pile-2 for code.
Check out the release blogpost here: https://huggingface.co/blog/EuroBERT/release
* EuroBERT/EuroBERT-210m
* EuroBERT/EuroBERT-610m
* EuroBERT/EuroBERT-2.1B
The next step is for researchers to build upon the 3 EuroBERT base models and publish strong retrieval, zero-shot classification, etc. models for all to use. I'm very much looking forward to it!
- 1 reply
liked
a
model
5 days ago
published
a
model
5 days ago
updated
a
model
5 days ago
upvoted
a
paper
6 days ago
updated
a
model
7 days ago
published
a
model
7 days ago
liked
a
model
7 days ago
liked
2
models
10 days ago
updated
a
model
10 days ago
published
a
model
10 days ago