Welcome PaliGemma 2 – New vision language models by Google
•
117
Merve Noyan
merve
AI & ML interests
VLMs, vision & co
Recent Activity
updated
a model
1 day ago
merve/colpali_ufo
updated
a model
3 days ago
HuggingFaceTB/SmolVLM-Instruct
new activity
3 days ago
HuggingFaceTB/SmolVLM-Instruct:Add FT tutorial link
Articles
Organizations
Posts
84
Post
1964
Aya by Cohere For AI can now see! 👀
C4AI community has built Maya 8B, a new open-source multilingual VLM built on SigLIP and Aya 8B 🌱 works on 8 languages! 🗣️
The authors extend Llava dataset using Aya's translation capabilities with 558k examples!
ry it here kkr5155/maya_demo
Dataset maya-multimodal/pretrain
Model maya-multimodal/maya 👏
kudos @nahidalam and team
C4AI community has built Maya 8B, a new open-source multilingual VLM built on SigLIP and Aya 8B 🌱 works on 8 languages! 🗣️
The authors extend Llava dataset using Aya's translation capabilities with 558k examples!
ry it here kkr5155/maya_demo
Dataset maya-multimodal/pretrain
Model maya-multimodal/maya 👏
kudos @nahidalam and team
Post
2558
Apollo is a new family of open-source video language models by Meta, where 3B model outperforms most 7B models and 7B outperforms most 30B models 🧶
✨ the models come in 1.5B https://huggingface.co/Apollo-LMMs/Apollo-1_5B-t32, 3B https://huggingface.co/Apollo-LMMs/Apollo-3B-t32 and 7B https://huggingface.co/Apollo-LMMs/Apollo-7B-t32 with A2.0 license, based on Qwen1.5 & Qwen2
✨ the authors also release a benchmark dataset https://huggingface.co/spaces/Apollo-LMMs/ApolloBench
The paper has a lot of experiments (they trained 84 models!) about what makes the video LMs work ⏯️
Try the demo for best setup here https://huggingface.co/spaces/Apollo-LMMs/Apollo-3B
they evaluate sampling strategies, scaling laws for models and datasets, video representation and more!
> The authors find out that whatever design decision was applied to small models also scale properly when the model and dataset are scaled 📈 scaling dataset has diminishing returns for smaller models
> They evaluate frame sampling strategies, and find that FPS sampling is better than uniform sampling, and they find 8-32 tokens per frame optimal
> They also compare image encoders, they try a variation of models from shape optimized SigLIP to DINOv2
they find google/siglip-so400m-patch14-384 to be most powerful 🔥
> they also compare freezing different parts of models, training all stages with some frozen parts give the best yield
They eventually release three models, where Apollo-3B outperforms most 7B models and Apollo 7B outperforms 30B models 🔥
✨ the models come in 1.5B https://huggingface.co/Apollo-LMMs/Apollo-1_5B-t32, 3B https://huggingface.co/Apollo-LMMs/Apollo-3B-t32 and 7B https://huggingface.co/Apollo-LMMs/Apollo-7B-t32 with A2.0 license, based on Qwen1.5 & Qwen2
✨ the authors also release a benchmark dataset https://huggingface.co/spaces/Apollo-LMMs/ApolloBench
The paper has a lot of experiments (they trained 84 models!) about what makes the video LMs work ⏯️
Try the demo for best setup here https://huggingface.co/spaces/Apollo-LMMs/Apollo-3B
they evaluate sampling strategies, scaling laws for models and datasets, video representation and more!
> The authors find out that whatever design decision was applied to small models also scale properly when the model and dataset are scaled 📈 scaling dataset has diminishing returns for smaller models
> They evaluate frame sampling strategies, and find that FPS sampling is better than uniform sampling, and they find 8-32 tokens per frame optimal
> They also compare image encoders, they try a variation of models from shape optimized SigLIP to DINOv2
they find google/siglip-so400m-patch14-384 to be most powerful 🔥
> they also compare freezing different parts of models, training all stages with some frozen parts give the best yield
They eventually release three models, where Apollo-3B outperforms most 7B models and Apollo 7B outperforms 30B models 🔥
spaces
106
models
89
merve/colpali_ufo
Updated
merve/paligemma_vqav2
Image-Text-to-Text
•
Updated
•
381
•
11
merve/paligemma2-3b-vqav2
Updated
•
293
•
2
merve/google-ckpts
Updated
merve/google-tokenizers
Updated
merve/idefics3-llama-vqav2
Updated
merve/idefics3llama-vqav2
Updated
•
8
merve/flux-dreambooth-lora
Updated
•
1
merve/trained-flux-lora-lego
Text-to-Image
•
Updated
•
10
•
•
1
merve/flux-lego-lora-dreambooth
Text-to-Image
•
Updated
•
1.63k
•
•
13
datasets
26
merve/model-test-inputs
Updated
•
34
merve/vqav2-small
Viewer
•
Updated
•
21.4k
•
1.76k
•
9
merve/SGinW
Viewer
•
Updated
•
16.7k
•
150
merve/pascal-voc
Viewer
•
Updated
•
336k
•
175
merve/YouCook2
Viewer
•
Updated
•
2k
•
58
merve/faiss_embeddings
Updated
•
8
merve/pokemon-ds-embeddings
Viewer
•
Updated
•
833
•
54
•
4
merve/tr-h4-norobots
Updated
•
68
•
10
merve/lego_sets_latest
Viewer
•
Updated
•
61
•
76
•
4
merve/ai-tube-dummy
Updated
•
43