Aymeric Roucher's picture

Aymeric Roucher

m-ric

·

http://aymeric-roucher.github.io

AI & ML interests

Leading Agents at Hugging Face 🤗

Recent Activity

liked a Space 3 days ago

data-agents/jupyter-agent

posted an update 3 days ago

After 6 years, BERT, the workhorse of encoder models, finally gets a replacement: 𝗪𝗲𝗹𝗰𝗼𝗺𝗲 𝗠𝗼𝗱𝗲𝗿𝗻𝗕𝗘𝗥𝗧! 🤗 We talk a lot about ✨Generative AI✨, meaning "Decoder version of the Transformers architecture", but this is only one of the ways to build LLMs: encoder models, that turn a sentence in a vector, are maybe even more widely used in industry than generative models. The workhorse for this category has been BERT since its release in 2018 (that's prehistory for LLMs). It's not a fancy 100B parameters supermodel (just a few hundred millions), but it's an excellent workhorse, kind of a Honda Civic for LLMs. Many applications use BERT-family models - the top models in this category cumulate millions of downloads on the Hub. ➡️ Now a collaboration between Answer.AI and LightOn just introduced BERT's replacement: ModernBERT. 𝗧𝗟;𝗗𝗥: 🏛️ Architecture changes: ⇒ First, standard modernizations: - Rotary positional embeddings (RoPE) - Replace GeLU with GeGLU, - Use Flash Attention 2 ✨ The team also introduced innovative techniques like alternating attention instead of full attention, and sequence packing to get rid of padding overhead. 🥇 As a result, the model tops the game of encoder models: It beats previous standard DeBERTaV3 for 1/5th the memory footprint, and runs 4x faster! Read the blog post 👉 https://huggingface.co/blog/modernbert

updated a Space 3 days ago

m-ric/ai-travel-planner

View all activity

Articles

Expert Support case study: Bolstering a RAG app with LLM-as-a-Judge

Our Transformers Code Agent beats the GAIA benchmark!

Extracting Concepts from LLMs: Anthropic’s recent discoveries 📖

License to Call: Introducing Transformers Agents 2.0

Open-source LLMs as LangChain Agents

Organizations

m-ric's activity

New activity in rhymes-ai/Aria 11 days ago

Upload processor

#14 opened 11 days ago by

Upload processor

#13 opened 11 days ago by

New activity in rhymes-ai/Aria-torchao-int8wo 13 days ago

Upload processor

#3 opened 13 days ago by

Upload AriaForConditionalGeneration

#2 opened 13 days ago by

commented a paper 18 days ago

ShowUI: One Vision-Language-Action Model for GUI Visual Agent

Paper • 2411.17465 • Published 26 days ago • 76 •

New activity in rhymes-ai/Aria 23 days ago

Upload processor

#12 opened 23 days ago by

Upload AriaForConditionalGeneration

#11 opened 23 days ago by

New activity in AtlaAI/judge-arena about 1 month ago

What are Meta-Llama-3.1-Instruct "Turbo" models?

#4 opened about 1 month ago by

commented a paper about 1 month ago

AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents

Paper • 2410.24024 • Published Oct 31 • 48 •

New activity in xinsir/controlnet-union-sdxl-1.0 about 2 months ago

is there a plan to support FLUX.1-schnell?

#28 opened 5 months ago by

commented a paper about 2 months ago

CLEAR: Character Unlearning in Textual and Visual Modalities

Paper • 2410.18057 • Published Oct 23 • 200 •

commented 3 papers 2 months ago

Differential Transformer

Paper • 2410.05258 • Published Oct 7 • 167 •

Aria: An Open Multimodal Native Mixture-of-Experts Model

Paper • 2410.05993 • Published Oct 8 • 107 •

Addition is All You Need for Energy-efficient Language Models

Paper • 2410.00907 • Published Oct 1 • 144 •

commented 4 papers 3 months ago

Were RNNs All We Needed?

Paper • 2410.01201 • Published Oct 2 • 47 •

Emu3: Next-Token Prediction is All You Need

Paper • 2409.18869 • Published Sep 27 • 92 •

Prithvi WxC: Foundation Model for Weather and Climate

Paper • 2409.13598 • Published Sep 20 • 39 •

YesBut: A High-Quality Annotated Multimodal Dataset for evaluating Satire Comprehension capability of Vision-Language Models

Paper • 2409.13592 • Published Sep 20 • 48 •

commented a paper 4 months ago

Diffusion Models Are Real-Time Game Engines

Paper • 2408.14837 • Published Aug 27 • 121 •

New activity in m-ric/agent-data-analyst 5 months ago

Interesting work appreciate keeping it open source

#1 opened 5 months ago by