Qwen2.5 Technical Report
Paper
•
2412.15115
•
Published
•
290
Learning UnkNown librAry
AI & ML interests
None defined yet.
Recent Activity
View all activity
luna-code's activity
huybery
authored
a
paper
3 days ago
huybery
authored
a
paper
11 days ago
SivilTaram
authored
3
papers
about 1 month ago
When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
Paper
•
2411.13476
•
Published
•
14
Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
Paper
•
2407.10956
•
Published
•
6
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
Paper
•
2411.04905
•
Published
•
111
SivilTaram
authored
a
paper
2 months ago
SivilTaram
authored
a
paper
3 months ago
huybery
authored
a
paper
3 months ago
huybery
authored
2
papers
4 months ago
huybery
authored
a
paper
5 months ago
SivilTaram
authored
a
paper
5 months ago
huybery
authored
a
paper
5 months ago
SivilTaram
posted
an
update
5 months ago
Post
2538
Still following your human intuition to mix corpora from different sources for pre-training 🧠? Everyone says that data mixture has a big impact on model performance, but how - and why🕵️? Did you know that web corpora are actually highly impactful for downstream tasks 🏆?
Check out our preprint "RegMix: Data Mixture as Regression for Language Model Pre-training" 📄
🔬 In this paper, we've proposed an automatic data mixture method RegMix that achieves a 6.3% improvement over human selection on the widely used HellaSwag benchmark - and it only needs a 2% extra training FLOPs! 📈
📄 Paper: RegMix: Data Mixture as Regression for Language Model Pre-training (2407.01492)
💻 Code: https://github.com/sail-sg/regmix
📊 Collection: sail/regmix-data-mixture-as-regression-6682b6caab37b9442877f0ce
🎮 Demo: https://huggingface.co/spaces/sail/RegMix
Check out our preprint "RegMix: Data Mixture as Regression for Language Model Pre-training" 📄
🔬 In this paper, we've proposed an automatic data mixture method RegMix that achieves a 6.3% improvement over human selection on the widely used HellaSwag benchmark - and it only needs a 2% extra training FLOPs! 📈
📄 Paper: RegMix: Data Mixture as Regression for Language Model Pre-training (2407.01492)
💻 Code: https://github.com/sail-sg/regmix
📊 Collection: sail/regmix-data-mixture-as-regression-6682b6caab37b9442877f0ce
🎮 Demo: https://huggingface.co/spaces/sail/RegMix
SivilTaram
authored
3
papers
6 months ago
RegMix: Data Mixture as Regression for Language Model Pre-training
Paper
•
2407.01492
•
Published
•
35
Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
Paper
•
2406.09136
•
Published
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Paper
•
2406.15877
•
Published
•
45
terryyz
authored
a
paper
6 months ago
SivilTaram
authored
2
papers
6 months ago