AlbertShi

AlbertShi's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

liked a model 14 days ago

SmallDoge/Doge-160M-Instruct

Question Answering • Updated 7 days ago • 60.4k • 9

upvoted a paper about 1 month ago

Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture

Paper • 2412.11834 • Published Dec 16, 2024 • 7

liked 4 models about 1 month ago

SmallDoge/Doge-20M-checkpoint

Text Generation • Updated 17 days ago • 136 • 4

SmallDoge/Doge-60M-checkpoint

Text Generation • Updated 17 days ago • 121 • 3

SmallDoge/Doge-160M-checkpoint

Text Generation • Updated 17 days ago • 195 • 3

SmallDoge/Doge-20M-Instruct

Question Answering • Updated 7 days ago • 8.61k • 3

upvoted a collection about 1 month ago

Doge

Collection

Doge family of small language models. • 8 items • Updated 12 days ago • 6

liked 3 models about 1 month ago

SmallDoge/Doge-60M-Instruct

Question Answering • Updated 7 days ago • 14.7k • 5

SmallDoge/Doge-20M

Text Generation • Updated 7 days ago • 8.2k • 8

SmallDoge/Doge-60M

Text Generation • Updated 7 days ago • 8.29k • 4

reacted to JingzeShi's post with 👍🤯👀 about 1 month ago

view post

Post

2076

Only a single RTX 4090 running model pre-training is really slow, even for small language models!!! (https://huggingface.co/collections/JingzeShi/doge-slm-677fd879f8c4fd0f43e05458)

See translation

• 
• 

• 
• 
• 2 replies

·

👀

8

8

🤯

6

6

👍

4

4

+

Reply

reacted to JingzeShi's post with 🔥 about 1 month ago

view post

Post

1712

🤩warmup -> stable -> decay leanring rate scheduler:
😎use the Stable Phase CheckPoints to Continue Training the model on Any New Dataset without spikes of the training!!!
SmallDoge/Doge-20M-checkpoint
SmallDoge/Doge-60M-checkpoint

See translation

• 

• 
• 
• 
• 4 replies

·

🔥

7

7

😎

1

1

👀

1

1

🤗

1

1

+

Reply