Shivalika Singh's picture

Shivalika Singh

shivi

·

shivalikasingh95

AI & ML interests

None yet

Recent Activity

reacted to dvilasuero's post with ❤️ 13 days ago

🌐 Announcing Global-MMLU: an improved MMLU Open dataset with evaluation coverage across 42 languages, built with Argilla and the Hugging Face community. Global-MMLU is the result of months of work with the goal of advancing Multilingual LLM evaluation. It's been an amazing open science effort with collaborators from Cohere For AI, Mila - Quebec Artificial Intelligence Institute, EPFL, Massachusetts Institute of Technology, AI Singapore, National University of Singapore, KAIST, Instituto Superior Técnico, Carnegie Mellon University, CONICET, and University of Buenos Aires. 🏷️ +200 contributors used Argilla MMLU questions where regional, dialect, or cultural knowledge was required to answer correctly. 85% of the questions required Western-centric knowledge! Thanks to this annotation process, the open dataset contains two subsets: 1. 🗽 Culturally Agnostic: no specific regional, cultural knowledge is required. 2. ⚖️ Culturally Sensitive: requires dialect, cultural knowledge or geographic knowledge to answer correctly. Moreover, we provide high quality translations of 25 out of 42 languages, thanks again to the community and professional annotators leveraging Argilla on the Hub. I hope this will ensure a better understanding of the limitations and challenges for making open AI useful for many languages. Dataset: https://huggingface.co/datasets/CohereForAI/Global-MMLU

reacted to dvilasuero's post with 🔥 13 days ago

🌐 Announcing Global-MMLU: an improved MMLU Open dataset with evaluation coverage across 42 languages, built with Argilla and the Hugging Face community. Global-MMLU is the result of months of work with the goal of advancing Multilingual LLM evaluation. It's been an amazing open science effort with collaborators from Cohere For AI, Mila - Quebec Artificial Intelligence Institute, EPFL, Massachusetts Institute of Technology, AI Singapore, National University of Singapore, KAIST, Instituto Superior Técnico, Carnegie Mellon University, CONICET, and University of Buenos Aires. 🏷️ +200 contributors used Argilla MMLU questions where regional, dialect, or cultural knowledge was required to answer correctly. 85% of the questions required Western-centric knowledge! Thanks to this annotation process, the open dataset contains two subsets: 1. 🗽 Culturally Agnostic: no specific regional, cultural knowledge is required. 2. ⚖️ Culturally Sensitive: requires dialect, cultural knowledge or geographic knowledge to answer correctly. Moreover, we provide high quality translations of 25 out of 42 languages, thanks again to the community and professional annotators leveraging Argilla on the Hub. I hope this will ensure a better understanding of the limitations and challenges for making open AI useful for many languages. Dataset: https://huggingface.co/datasets/CohereForAI/Global-MMLU

updated a dataset 15 days ago

CohereForAI/Global-MMLU-Lite

View all activity

Articles

A Deepdive into Aya Expanse: Advancing the Frontier of Multilinguality

Universal Image Segmentation with Mask2Former and OneFormer

Organizations

shivi's activity

New activity in CohereForAI/Global-MMLU 23 days ago

Quality Issues with JA dev set

#5 opened 25 days ago by

Librarian Bot: Add language metadata for dataset

#3 opened 28 days ago by

Include argilla tag

#2 opened 28 days ago by

Duplicates for NL

#4 opened 26 days ago by

New activity in CohereForAI/Global-MMLU 28 days ago

[bot] Conversion to Parquet

#1 opened 28 days ago by

parquet-converter

New activity in CohereForAI/aya_expanse 2 months ago

refactor: simplify text validation

#2 opened 2 months ago by

Single T4

#1 opened 2 months ago by

New activity in CohereForAI/aya-expanse-8b 2 months ago

ValueError: Tokenizer class CohereTokenizer does not exist or is not currently imported.

#9 opened 2 months ago by

how did you use merging?

#7 opened 2 months ago by

First impression: Impressed

#2 opened 2 months ago by

New activity in CohereForAI/aya-expanse-32b 2 months ago

Context Length Set to 8K?

#4 opened 2 months ago by

GQA?

#3 opened 2 months ago by

PlatonicSkeptic

New activity in CohereForAI/aya-expanse-8b 2 months ago

Thank you for implement Czech language!

#11 opened 2 months ago by

New activity in CohereForAI/aya-expanse-32b 2 months ago

Comparing with Command R-plus-08-2024

#2 opened 2 months ago by

New activity in CohereForAI/aya-expanse-8b 2 months ago

Adding Evaluation Results

#3 opened 2 months ago by

leaderboard-pr-bot

Any chance of a 1B/2B/3B/4B model?

#5 opened 2 months ago by

apply_chat_template with tools not working

#4 opened 2 months ago by

Models appears to be 'too safe', likes to lecture.

#1 opened 2 months ago by

New activity in CohereForAI/aya-expanse-32b 2 months ago

Wrong "max_position_embeddings" value?

#1 opened 2 months ago by

New activity in CohereForAI/aya_collection_language_split 4 months ago

A lot of german examples from MLQA-en TEST dataset don't have a question with its passage

#5 opened 4 months ago by