Xiaosen Zheng's picture

Xiaosen Zheng

xszheng2020

·

AI & ML interests

Data-Centric AI and AI Safety.

Organizations

xszheng2020's activity

upvoted an article about 21 hours ago

Article

SmolLM - blazingly fast and remarkably powerful

Jul 16

• 248

upvoted a paper 4 days ago

Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling

Paper • 2401.16380 • Published Jan 29 • 47

upvoted an article 9 days ago

Article

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Mar 20

• 62

upvoted a paper 11 days ago

Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates

Paper • 2410.07137 • Published 11 days ago • 6

upvoted a paper 22 days ago

Programming Every Example: Lifting Pre-training Data Quality like Experts at Scale

Paper • 2409.17115 • Published 25 days ago • 59

upvoted a paper about 2 months ago

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler

Paper • 2408.13359 • Published Aug 23 • 21

upvoted a collection about 2 months ago

Power-LM

Dense & MoE LLMs trained with power learning rate scheduler. • 4 items • Updated 3 days ago • 15

upvoted a paper 3 months ago

MiniCPM-V: A GPT-4V Level MLLM on Your Phone

Paper • 2408.01800 • Published Aug 3 • 75

upvoted 2 collections 3 months ago

Llama 3.1

This collection hosts the transformers and original repos of the Llama 3.1, Llama Guard 3 and Prompt Guard models • 11 items • Updated 25 days ago • 596

Model with Circuit Breakers

SoTA models with circuit breakers inserted. Top safety performance without losing capabilities. • 2 items • Updated Jul 9 • 3

upvoted a paper 3 months ago

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies

Paper • 2407.13623 • Published Jul 18 • 52

upvoted 2 articles 3 months ago

Article

How NuminaMath Won the 1st AIMO Progress Prize

Jul 11

• 95

Article

RegMix: Data Mixture as Regression for Language Model Pre-training

By

•

Jul 11

• 10

upvoted a paper 4 months ago

RegMix: Data Mixture as Regression for Language Model Pre-training

Paper • 2407.01492 • Published Jul 1 • 33

upvoted a collection 4 months ago

🧬 RegMix: Data Mixture as Regression

Automatic data mixture method for large language model pre-training • 10 items • Updated Jul 26 • 5

upvoted 3 papers 4 months ago

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

Paper • 2406.11931 • Published Jun 17 • 57

Bootstrapping Language Models with DPO Implicit Rewards

Paper • 2406.09760 • Published Jun 14 • 37

Scaling Data-Constrained Language Models

Paper • 2305.16264 • Published May 25, 2023 • 17

upvoted 6 papers 5 months ago

An Empirical Study of Memorization in NLP

Paper • 2203.12171 • Published Mar 23, 2022 • 1

Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast

Paper • 2402.08567 • Published Feb 13 • 2

Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses

Paper • 2406.01288 • Published Jun 3 • 1

Intriguing Properties of Data Attribution on Diffusion Models

Paper • 2311.00500 • Published Nov 1, 2023 • 2

LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition

Paper • 2307.13269 • Published Jul 25, 2023 • 31

Zero Bubble Pipeline Parallelism

Paper • 2401.10241 • Published Nov 30, 2023 • 22

upvoted a collection 8 months ago

⚓️ Sailor Language Models

Sailor: Open Language Models tailored for South-East Asia (SEA) released by Sea AI Lab. • 18 items • Updated Jul 26 • 16