Hugging Face's logo
nicolo's picture
28 10 60

nicolo

nicolollo
21world's profile picture
·

AI & ML interests

None yet

Recent Activity

liked a Space 5 days ago
fancyfeast/joy-caption-alpha-two
liked a model 25 days ago
nomic-ai/nomic-embed-text-v2-moe
reacted to grimjim's post with ❤️ 27 days ago
This recent paper points to an explanation for the unreasonable effectiveness of Frankenmerges: https://huggingface.co/papers/2502.05171 Specifically, the duplication of layers in Frankenmerges serves a purpose similar to what occurs in their recurrent-depth architecture. Successful frankenmerges that operate without additional fine-tuning are able to recover or "heal" from any damage due to abrupt transitions between layer blocks. Operational replicated layer blocks can provide functional benefits grounded in latent reasoning. Frankenmerges can also result in hybrid reasoning, by splicing together the latent reasoning of different models. Back in April 2024, I was able to duplicate a few layers in the Llama 3 8B model, turning it into a 9B model, without harming benchmarks significantly, despite any transition damage. https://huggingface.co/grimjim/llama-3-experiment-v1-9B My informal experimentation suggested that latent reasoning circuits could occupy continguous stacks of 2-4 layers, though the result was highly sensitive to the choice of transition location between layers.
View all activity

Organizations

Hugging Face Discord Community's profile picture

nicolollo's activity

upvoted 2 collections about 2 months ago
upvoted a collection 2 months ago
upvoted a collection 4 months ago
upvoted an article 5 months ago
view article
Article

🇮🇹🇯🇵🇧🇷 Generating multilingual instruction datasets with Magpie 🐦‍⬛

By anakin87 •
• 19
upvoted a collection 6 months ago
upvoted an article 8 months ago
upvoted a collection 8 months ago
upvoted an article 8 months ago