Note The Aya Vision Benchmark is designed to evaluate vision-language models in real-world multilingual scenarios. It spans 23 languages and 9 distinct task categories, with 15 samples per category, resulting in 135 image-question pairs per language.

Note The m-WildVision dataset is a multilingual multimodal LLM evaluation set covering 23 languages. It was created by translating prompts from the original English-only WildVision test set. The original prompts, developed by Lu et al. (2024) , consist of 500 challenging user queries sourced from the WildVision-Arena platform.