Article
makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch
By
•
•
62Mayank Bhaskar
cataluna84
AI & ML interests
Research interest includes anything that moves or in processing things that are on the audible spectrum and has recently been seduced by RL Theory, GNNs, Transformers & applications of them in solving self-driving autonomous systems including vehicles, drones, and robots.
Recent Activity
upvoted
an
article
1 day ago
makeMoE: Implement a Sparse Mixture of Experts Language Model from Scratch
upvoted
an
article
1 day ago
Mixture of Experts Explained
Organizations
cataluna84's activity
upvoted
3
articles
1 day ago
Article
Sparse Mixture of Experts Language Model from Scratch: Extending makeMoE with Expert Capacity
By
•
•
10Article
Mixture of Experts Explained
•
459
liked
9
models
1 day ago
google/shieldgemma-2-4b-it
Image-Text-to-Text
•
Updated
•
682
•
49
google/gemma-3-27b-it
Image-Text-to-Text
•
Updated
•
190k
•
667
google/gemma-3-27b-pt
Image-Text-to-Text
•
Updated
•
4.26k
•
61
google/gemma-3-12b-it
Image-Text-to-Text
•
Updated
•
65.6k
•
204
google/gemma-3-12b-pt
Image-Text-to-Text
•
Updated
•
2.23k
•
30
google/gemma-3-1b-it
Text Generation
•
Updated
•
49.9k
•
158
google/gemma-3-1b-pt
Text Generation
•
Updated
•
15.3k
•
67
google/gemma-3-4b-pt
Image-Text-to-Text
•
Updated
•
6.12k
•
44
google/gemma-3-4b-it
Image-Text-to-Text
•
Updated
•
79.8k
•
210
upvoted
a
collection
2 days ago
upvoted
an
article
2 days ago
Article
Welcome Gemma 3: Google's all new multimodal, multilingual, long context open LLM
•
278
liked
a
model
3 days ago
liked
5
models
11 days ago
Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int8
Image-Text-to-Text
•
Updated
•
1.8k
•
11
Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4
Image-Text-to-Text
•
Updated
•
35.1k
•
27
Qwen/Qwen2-VL-72B-Instruct-AWQ
Image-Text-to-Text
•
Updated
•
10.2k
•
48
Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int8
Image-Text-to-Text
•
Updated
•
3.32k
•
29
Qwen/Qwen2-VL-7B-Instruct-GPTQ-Int4
Image-Text-to-Text
•
Updated
•
41.5k
•
35