Datasets: NeurIPS LLM Challenge 2023
Datasets that were under consideration for usage in my submission to the 2023 NeurIPS Large Language Model Efficiency Challenge.
Viewer • Updated • 63k • 451 • 32Note Ultimately used in my full eval submission, with exclusion of dolly_hhrlhf. Included only in Mistral-7B-sft-v1.
databricks/databricks-dolly-15k
Viewer • Updated • 15k • 10.3k • 752Note Used both for Mistral-7B-sft-v0 and Mistral-7B-sft-v1 in my submissions.
hendrycks/competition_math
Updated • 43.5k • 121Note This turned out to be one of the holdout tasks. mosaicml/instruct-v3 includes training problems from competition_math. This is a contributor to the high scores on GSM8k and MATH benchmarks.
kaist-ai/CoT-Collection
Viewer • Updated • 1.84M • 1.06k • 116Note Looked promising, but did not have time to explore.
tasksource/icl-symbol-tuning-instruct
Viewer • Updated • 484k • 156 • 18Note Considered for improving ICL. Did not have time to explore.
cais/mmlu
Viewer • Updated • 231k • 62.2k • 319Note Decided against training on MMLU data.
GAIR/lima
Viewer • Updated • 1.33k • 1.15k • 410Note Avoided due to CC BY-NC-SA license, though it would have been allowed for the competition. Likely would have been a good resource otherwise.
grammarly/coedit
Viewer • Updated • 70.8k • 895 • 61Note The plan here would be to target robustness metrics by finetuning an expert model to correct perturbations and/or clarify the input. This could have paraphrasing or other text revision tasks if they appeared in the hidden eval. Did not have time to fully explore.
wanyu/IteraTeR_human_sent
Viewer • Updated • 4.02k • 199Note Similar use case as coedit.
allenai/social_i_qa
Updated • 14.5k • 13Note Now knowing that the holdout tasks had ethics questions, I wish I had used this.
lighteval/siqa
Viewer • Updated • 35.4k • 5.98k • 4Note Same as social_i_qa
tau/commonsense_qa
Viewer • Updated • 12.1k • 19.1k • 72Note Now knowing that the holdout tasks had ethics questions, I wish I had used this.
euirim/goodwiki
Viewer • Updated • 44.8k • 136 • 48Note Could have been useful for RAG.
alexfabbri/multi_news
Updated • 2.49k • 55Note The thought was this could help with CNN/DM summarization, but some quality and license concerns combined with acceptable performance without it led to its exclusion.
allenai/math_qa
Updated • 46.9k • 84allenai/ropes
Viewer • Updated • 14.3k • 266 • 45allenai/openbookqa
Viewer • Updated • 11.9k • 36.5k • 77allenai/ai2_arc
Viewer • Updated • 7.79k • 110k • 140INK-USC/riddle_sense
Updated • 132 • 21allenai/qasc
Viewer • Updated • 9.98k • 875 • 14nyu-mll/blimp
Viewer • Updated • 67k • 7.4k • 34google/boolq
Viewer • Updated • 12.7k • 6.57k • 62corypaik/prost
Viewer • Updated • 18.7k • 186 • 1allenai/sciq
Viewer • Updated • 13.7k • 9.82k • 90facebook/belebele
Viewer • Updated • 110k • 8.35k • 98derek-thomas/ScienceQA
Viewer • Updated • 21.2k • 2.5k • 145openlifescienceai/medmcqa
Viewer • Updated • 193k • 2.82k • 120embedding-data/QQP_triplets
Viewer • Updated • 102k • 250 • 7VMware/open-instruct
Viewer • Updated • 143k • 288 • 44