Open Science

open-science's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts

eliebak 

posted an update 7 months ago

view post

Post

1151

Wow, impressive 340B model by nvidia with a nice permissive license! 🚀 The technical report is full of insights and seems to use a different learning rate schedule than cosine, probably a variant of WSD. Hope to get more info on that! 👀

nvidia/nemotron-4-340b-666b7ebaf1b3867caf2f1911

• 
• 

🤗

1

1

+

Reply

eliebak 

authored a paper 7 months ago

Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations

Paper • 2405.18392 • Published May 28, 2024 • 12