iVideoGPT: Interactive VideoGPTs are Scalable World Models
Paper
•
2405.15223
•
Published
•
11
multimodal
-
-
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models
Paper • 2405.15574 • Published • 52 -
An Introduction to Vision-Language Modeling
Paper • 2405.17247 • Published • 77 -
Matryoshka Multimodal Models
Paper • 2405.17430 • Published • 29 -
Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities
Paper • 2405.18669 • Published • 11 -
MotionLLM: Understanding Human Behaviors from Human Motions and Videos
Paper • 2405.20340 • Published • 19 -
Parrot: Multilingual Visual Instruction Tuning
Paper • 2406.02539 • Published • 35 -
PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM
Paper • 2406.02884 • Published • 13 -
What If We Recaption Billions of Web Images with LLaMA-3?
Paper • 2406.08478 • Published • 38 -
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
Paper • 2406.07476 • Published • 30 -
MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
Paper • 2406.08407 • Published • 23 -
AV-DiT: Efficient Audio-Visual Diffusion Transformer for Joint Audio and Video Generation
Paper • 2406.07686 • Published • 13 -
OpenVLA: An Open-Source Vision-Language-Action Model
Paper • 2406.09246 • Published • 28 -
Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models
Paper • 2406.09403 • Published • 17 -
Explore the Limits of Omni-modal Pretraining at Scale
Paper • 2406.09412 • Published • 10 -
4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities
Paper • 2406.09406 • Published • 12 -
ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation
Paper • 2406.09961 • Published • 53 -
Needle In A Multimodal Haystack
Paper • 2406.07230 • Published • 51 -
OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text
Paper • 2406.08418 • Published • 28 -
mDPO: Conditional Preference Optimization for Multimodal Large Language Models
Paper • 2406.11839 • Published • 35 -
LLaNA: Large Language and NeRF Assistant
Paper • 2406.11840 • Published • 17 -
Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs
Paper • 2406.14544 • Published • 33 -
PIN: A Knowledge-Intensive Dataset for Paired and Interleaved Multimodal Documents
Paper • 2406.13923 • Published • 20 -
Improving Visual Commonsense in Language Models via Multiple Image Generation
Paper • 2406.13621 • Published • 13 -
Multimodal Structured Generation: CVPR's 2nd MMFM Challenge Technical Report
Paper • 2406.11403 • Published • 4 -
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters
Paper • 2406.16758 • Published • 16 -
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs
Paper • 2406.16860 • Published • 45 -
Long Context Transfer from Language to Vision
Paper • 2406.16852 • Published • 30 -
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
Paper • 2406.15704 • Published • 5