Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
Arabic MMLU
#9
by
Malikeh1375
- opened
Hi,
There are two versions of MMLU: one is the Arabic MMLU one generated by arabic speakers (https://huggingface.co/datasets/OALL/Arabic_MMLU) and the other one is the translated version of MMLU (https://huggingface.co/datasets/OALL/AlGhafa-Arabic-LLM-Benchmark-Translated). I wonder which one is used for the evaluation part of OALL.
Best,
Malikeh
Hey there,
Well first this one (https://huggingface.co/datasets/OALL/Arabic_MMLU) is not generated by arabic speakers but translated also by AceGPT team ! The MMLU reported in the leaderboard is that one (https://huggingface.co/datasets/OALL/Arabic_MMLU). While we use other subsets from (https://huggingface.co/datasets/OALL/AlGhafa-Arabic-LLM-Benchmark-Translated) but MMLU
Thank you
alielfilali01
changed discussion status to
closed