Sourced from Wikimedia - a davanstrien Collection

davanstrien 's Collections

synthetic-data-generation-demos

sentence-transformers-from-synthetic-data

Synthetic (text) Dataset Generation

haiku

Historic language modeling

Climate

Sourced from Wikimedia

Legal Named Entity Recognition

Top 10% instruction tuning datasets

Top 10 Instruction Tuning Datasets copy

Metadata-generation

MOE papers to read

German Text Embedding Clustering Benchmark datasets

cosmochat-reading-list

datasets-tldr-project

Probably DPO datasets

Probably Alpaca Style Datasets

Direct Preference Optimization Datasets

Image Preference Optimization Datasets

query-to-hub-datasets-viewer-project

Sourced from Wikimedia

updated Jun 21

Wikimedia collections, i.e. Wikipedia, are heavily used in ML research. This collection highlights some prominent examples of these datasets.

legacy-datasets/wikipedia

Updated Mar 11 • 28.1k • 556

Note Wikipedia dataset containing cleaned articles of all languages.
Salesforce/wikitext

Viewer • Updated Jan 4 • 3.71M • 373k • 363

Note The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified Good and Featured articles on Wikipedia.
EleutherAI/wikitext_document_level

Updated Apr 5 • 22.1k • 12

Note This is a modified version of https://huggingface.co/datasets/wikitext that returns Wiki pages instead of Wiki text line-by-line.
Salesforce/wikisql

Updated Jan 18 • 1.16k • 100
microsoft/wiki_qa

Viewer • Updated Jan 4 • 29.3k • 4.54k • 45
unimelb-nlp/wikiann

Viewer • Updated Feb 22 • 2M • 61.5k • 102
Cohere/wikipedia-22-12-simple-embeddings

Viewer • Updated Mar 22, 2023 • 486k • 477 • 55
chaojiang06/wiki_auto

Updated Jan 18 • 121 • 8
euirim/goodwiki

Viewer • Updated Sep 11, 2023 • 44.8k • 115 • 49