Probably DPO datasets

davanstrien 's Collections

synthetic-data-generation-demos

sentence-transformers-from-synthetic-data

Synthetic (text) Dataset Generation

haiku

Historic language modeling

Climate

Sourced from Wikimedia

Legal Named Entity Recognition

Top 10% instruction tuning datasets

Top 10 Instruction Tuning Datasets copy

Metadata-generation

MOE papers to read

German Text Embedding Clustering Benchmark datasets

cosmochat-reading-list

datasets-tldr-project

Probably DPO datasets

Probably Alpaca Style Datasets

Direct Preference Optimization Datasets

Image Preference Optimization Datasets

query-to-hub-datasets-viewer-project

updated Jun 26

A collection of datasets that probably support DPO

Upvote

HuggingFaceH4/ultrafeedback_binarized

Viewer • Updated Oct 16 • 187k • 6.84k • 254
mlabonne/orpo-dpo-mix-40k

Viewer • Updated Oct 17 • 44.2k • 1.44k • 261
argilla/OpenHermesPreferences

Viewer • Updated Mar 1 • 989k • 792 • 201
argilla/distilabel-capybara-dpo-7k-binarized

Viewer • Updated Jul 16 • 7.56k • 3.25k • 177
argilla/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 5 • 12.9k • 733 • 168
argilla/dpo-mix-7k

Viewer • Updated Jul 16 • 7.5k • 659 • 156
argilla/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 60.9k • 9.24k • 127
Dahoas/rm-static

Viewer • Updated Mar 6, 2023 • 81.4k • 7.07k • 112
openbmb/UltraInteract_pair

Viewer • Updated Apr 5 • 220k • 441 • 105
openbmb/RLAIF-V-Dataset

Viewer • Updated Nov 3 • 33.8k • 1.6k • 131
llamafactory/DPO-En-Zh-20k

Viewer • Updated Jun 7 • 20k • 209 • 90
Dahoas/full-hh-rlhf

Viewer • Updated Feb 23, 2023 • 125k • 1.62k • 74
argilla/OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Feb 10 • 9.79k • 50 • 64
yitingxie/rlhf-reward-datasets

Viewer • Updated Jan 1, 2023 • 81.4k • 811 • 62
allenai/ultrafeedback_binarized_cleaned

Viewer • Updated Dec 1, 2023 • 186k • 1.29k • 69
allenai/reward-bench

Viewer • Updated Sep 9 • 8.11k • 6.18k • 80
CarperAI/openai_summarize_comparisons

Viewer • Updated Feb 27, 2023 • 260k • 1.78k • 40
beyond/rlhf-reward-single-round-trans_chinese

Viewer • Updated Jul 5, 2023 • 24.9k • 67 • 37
argilla/Capybara-Preferences

Viewer • Updated May 9 • 15.4k • 246 • 40
snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset

Viewer • Updated Jan 23 • 62.7k • 111 • 42
tasksource/oasst1_pairwise_rlhf_reward

Viewer • Updated Jul 4, 2023 • 18.9k • 119 • 42
wenbopan/Chinese-dpo-pairs

Viewer • Updated Apr 2 • 10.7k • 94 • 39
maywell/ko_Ultrafeedback_binarized

Viewer • Updated Nov 9, 2023 • 62k • 103 • 29
abacusai/MetaMath_DPO_FewShot

Viewer • Updated Feb 26 • 395k • 108 • 26
HuggingFaceH4/orca_dpo_pairs

Viewer • Updated Apr 14 • 12.9k • 152 • 25
Dahoas/static-hh

Viewer • Updated Mar 6, 2023 • 101k • 83 • 20
mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Mar 21 • 9.2k • 49 • 19
HuggingFaceH4/grok-conversation-harmless

Viewer • Updated Feb 2 • 44.8k • 56 • 21
allenai/preference-test-sets

Viewer • Updated Mar 14 • 43.2k • 433 • 21
abacusai/ARC_DPO_FewShot

Viewer • Updated Feb 26 • 4.25k • 73 • 17
cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental

Viewer • Updated Aug 15 • 29.2k • 178 • 19
vicgalle/configurable-system-prompt-multitask

Viewer • Updated Apr 23 • 1.95k • 156 • 19
coseal/CodeUltraFeedback_binarized

Viewer • Updated Mar 18 • 9.5k • 751 • 15
NobodyExistsOnTheInternet/ToxicDPOqa

Viewer • Updated Apr 26 • 6.87k • 53 • 15
multilingual/orca_dpo_pairs

Viewer • Updated Apr 3 • 64.7k • 141 • 15
fblgit/simple-math-DPO

Viewer • Updated Aug 1 • 800k • 117 • 16
HuggingFaceH4/cai-conversation-harmless

Viewer • Updated Feb 2 • 44.8k • 230 • 14
alvarobartt/dpo-mix-7k-simplified

Viewer • Updated Mar 26 • 7.5k • 56 • 11
kyujinpy/orca_math_dpo

Viewer • Updated Apr 12 • 15.3k • 70 • 18
davanstrien/dataset-tldr-preference-dpo

Viewer • Updated Apr 30 • 522 • 50 • 11
trl-internal-testing/hh-rlhf-trl-style

Viewer • Updated Mar 13 • 169k • 542 • 11
abacusai/HellaSwag_DPO_FewShot

Viewer • Updated Feb 26 • 150k • 84 • 8
2A2I/argilla-dpo-mix-7k-arabic

Viewer • Updated Mar 18 • 7.5k • 63 • 9
argilla/Capybara-Preferences-Filtered

Viewer • Updated Apr 20 • 14.8k • 68 • 9
2A2I/Aya-Command.R-DPO

Viewer • Updated May 16 • 14.2k • 37 • 7
kuotient/orca-math-korean-preference

Viewer • Updated Apr 10 • 193k • 238 • 10
M4-ai/prm_dpo_pairs

Viewer • Updated Jul 1 • 93.9k • 51 • 7
zerolink/zsql-postgres-dpo

Viewer • Updated Feb 2 • 259k • 66 • 6
princeton-nlp/llama3-ultrafeedback

Viewer • Updated Jul 18 • 61.8k • 1.05k • 16
kaist-ai/Multifaceted-Collection-ORPO

Viewer • Updated Jul 1 • 64.6k • 54 • 10
kaitchup/UltraFeedback-prompt-chosen-rejected

Viewer • Updated Oct 23, 2023 • 18k • 67 • 6
alexredna/oasst2_dpo_pairs

Viewer • Updated Jul 17 • 10k • 53 • 6
HuggingFaceH4/grok-conversation-harmless2

Viewer • Updated Jan 19 • 44.8k • 65 • 6
yongtae-jp/orca_dpo_pairs_ja

Viewer • Updated Jan 9 • 12.9k • 110 • 6
kaist-ai/Multifaceted-Collection-DPO

Viewer • Updated Jul 1 • 65.1k • 97 • 8
malhajar/distilabel-intel-orca-dpo-pairs-tr

Viewer • Updated Mar 28 • 9.12k • 51 • 6
efederici/alpaca-vs-alpaca-orpo-dpo

Viewer • Updated May 15 • 49.2k • 129 • 7
maywell/ko_Ultrafeedback_binarized_10k

Viewer • Updated Nov 4, 2023 • 10k • 39 • 5
kira/math-dpo

Viewer • Updated Jul 2 • 2.42k • 59 • 7
mlabonne/distilabel-truthy-dpo-v0.1

Viewer • Updated Feb 12 • 1.02k • 64 • 5
RLHFlow/UltraFeedback-preference-standard

Viewer • Updated Apr 27 • 340k • 94 • 8
BramVanroy/orca_dpo_pairs_dutch

Viewer • Updated 17 days ago • 11k • 74 • 5
argilla/ultrafeedback-multi-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 158k • 109 • 6
malhajar/orca_dpo_pairs-tr

Viewer • Updated Mar 30 • 10.8k • 90 • 6
lightblue/mitsu_tophalf_borda

Viewer • Updated May 30 • 1.42k • 58 • 5
NickyNicky/neovalle_H4rmony_dpo_translated_English_to_Spanish

Viewer • Updated May 17 • 2.02k • 48 • 4
maywell/ko_hh-rlhf-20k_filtered

Viewer • Updated Nov 4, 2023 • 19.4k • 40 • 4
kaist-ai/Multifaceted-Collection-RM

Viewer • Updated Jun 7 • 64.9k • 56 • 4
HuggingFaceH4/hh-rlhf-h4

Viewer • Updated Nov 29, 2023 • 169k • 163 • 4
orpo-explorers/ultrafeedback_binarized_15k

Viewer • Updated Apr 18 • 15.4k • 36 • 4
We-Want-GPU/Yi-Ko-DPO-Orca-DPO-Pairs

Viewer • Updated Dec 18, 2023 • 12.7k • 59 • 5
zerolink/zsql-sqlite-dpo

Viewer • Updated Feb 2 • 260k • 105 • 3
argilla/ultrafeedback-multi-binarized-quality-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 155k • 44 • 4
SAGI-1/Greetings_DPO_dataset_V1

Viewer • Updated Dec 18, 2023 • 891 • 43 • 3
2A2I/Aya-SambaLingo.Arabic.Chat-DPO

Viewer • Updated May 25 • 14.2k • 57 • 3
AlekseyKorshuk/evol-codealpaca-v1-dpo

Viewer • Updated Jan 25 • 39.9k • 58 • 4
mlabonne/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Mar 27 • 28k • 38 • 3
kuotient/orca-math-korean-dpo-pairs

Viewer • Updated Apr 5 • 193k • 117 • 9
mii-community/ultrafeedback-preferences-translated-ita

Viewer • Updated Feb 21 • 60.9k • 42 • 3
sqrti/SPA-VL

Viewer • Updated Jul 3 • 101k • 624 • 7
HuggingFaceH4/orca_dpo_pairs_no_system_prompt

Viewer • Updated Jan 15 • 12.9k • 39 • 3
mlabonne/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 6 • 5.92k • 38 • 3
d0rj/rlhf-reward-datasets-ru

Viewer • Updated Jul 2 • 81.4k • 101 • 4
RLHFlow/HH-RLHF-Harmless-and-RedTeam-standard

Viewer • Updated May 8 • 42.3k • 76 • 2
2A2I/Aya-AceGPT.13B.Chat-DPO

Viewer • Updated May 16 • 13k • 57 • 2
RLHFlow/Helpsteer-preference-standard

Viewer • Updated Apr 27 • 37.1k • 183 • 4
IlyaGusev/saiga_preferences

Viewer • Updated Oct 28 • 30.6k • 87 • 4
RLHFlow/CodeUltraFeedback-standard

Viewer • Updated Apr 27 • 50.2k • 58 • 4
RLHFlow/UltraInteract-filtered-standard

Viewer • Updated Apr 28 • 162k • 52 • 2
RLHFlow/Prometheus2-preference-standard

Viewer • Updated May 5 • 200k • 47 • 2
RLHFlow/PKU-SafeRLHF-30K-standard

Viewer • Updated Apr 29 • 26.9k • 52 • 2
RLHFlow/Argilla-Math-DPO-standard

Viewer • Updated Apr 30 • 2.42k • 52 • 2
droussis/UltraSafety_binarized-orpo-dpo

Viewer • Updated Apr 25 • 2.26k • 40 • 2
We-Want-GPU/Yi-Ko-DPO-dataset

Viewer • Updated Dec 9, 2023 • 4k • 33 • 4
roborovski/glaive-tool-usage-dpo

Viewer • Updated Feb 29 • 42k • 38 • 2
vicgalle/OpenHermesPreferences-roleplay

Viewer • Updated Feb 29 • 3.06k • 51 • 5
GrowingApple/orpo_kor_translated_medical

Viewer • Updated Jun 12 • 10.4k • 47 • 3
reciprocate/ultrafeedback_cleaned_high_dpo

Viewer • Updated Jan 11 • 40k • 35 • 2
mlabonne/distilabel-truthy-dpo-v0.1-filtered

Viewer • Updated Feb 15 • 663 • 39 • 2
xiaodongguaAIGC/awesome-dpo

Viewer • Updated Jul 3 • 302k • 56 • 2
jan-hq/evol_codealpaca_dpo_binarized

Viewer • Updated Feb 20 • 39.9k • 38 • 2
2A2I/Aya-Aya.23.8B-DPO

Viewer • Updated May 29 • 14k • 44 • 2
tasksource/oasst2_pairwise_rlhf_reward

Viewer • Updated Jan 9 • 28.4k • 44 • 2
NickyNicky/function_call_orpo_sft_phi3_chatML_only

Viewer • Updated May 2 • 66k • 52 • 2
HuggingFaceH4/summarize_from_feedback

Viewer • Updated Mar 11 • 353k • 46 • 2
weqweasdas/preference_dataset_mix2

Viewer • Updated Mar 21 • 528k • 55 • 3
NickyNicky/oasst2_orpo_mix_function_call_phi_3_v1

Viewer • Updated May 2 • 120k • 36 • 2
BramVanroy/ultra_feedback_dutch

Viewer • Updated 17 days ago • 53.6k • 182 • 2
orpo-explorers/OpenHermesPreferences-10k

Viewer • Updated Mar 30 • 10k • 49 • 2
orpo-explorers/OpenHermesPreferences-500k

Viewer • Updated Mar 30 • 500k • 39 • 2
zhengr/ultrafeedback_binarized

Viewer • Updated Nov 8, 2023 • 190k • 128 • 2
ibivibiv/cleaned_orca_math_dpo_pairs

Viewer • Updated May 24 • 200k • 61 • 2
NickyNicky/nano_finance_200k_en_es_chatML_gemma_orpo_dpo

Viewer • Updated May 29 • 201k • 49 • 1
Baidicoot/hh-rlhf-golden-harmful

Viewer • Updated May 10 • 7.64k • 34 • 1
Tachi67/rm_data_motivation

Viewer • Updated Jun 4 • 44.8k • 41 • 1
trl-internal-testing/hh-rlhf-helpful-base-trl-style

Viewer • Updated May 2 • 46.2k • 5.61k • 9
flyingfishinwater/ultrafeedback_clean

Viewer • Updated Jul 2 • 175k • 80 • 1
manishiitg/argilla-distilabel-intel-orca-dpo-pairs

Viewer • Updated Jan 29 • 23.6k • 44 • 1
Palash123/dpo_anthropic_hh_rlhf

Viewer • Updated Dec 4, 2023 • 168k • 53 • 1
David-Xu/astronomy-stack-dpo-text

Viewer • Updated Mar 8 • 19.9k • 39 • 2
d0rj/full-hh-rlhf-ru

Viewer • Updated Jul 2 • 125k • 55 • 2
vwxyzjn/summarize_from_feedback_oai_preprocessing_1706381144

Viewer • Updated Jan 27 • 179k • 206 • 2
reciprocate/ultrafeedback_orca_math_cleaned_high_dpo

Viewer • Updated Jan 11 • 48.3k • 36 • 1
HuggingFaceH4/OpenHermes-2.5-preferences-v0-deduped

Viewer • Updated Feb 9 • 802k • 42 • 1
mlabonne/truthy-dpo-v0.1

Viewer • Updated Feb 18 • 1.02k • 38 • 1
macadeliccc/distilabel-neurology-preferences-2k-orca-format

Viewer • Updated Feb 22 • 1.99k • 41 • 1
jan-hq/athirdpath_roleplay_dpo_binarized

Viewer • Updated Jan 9 • 3.43k • 42 • 1
reciprocate/ultrafeedback_orca_math_dpo

Viewer • Updated Jan 8 • 73.8k • 33 • 1
d0rj/synthetic-instruct-gptj-pairwise-ru

Viewer • Updated Jul 2 • 33.1k • 64 • 1
CorticalStack/tak-stack-dpo

Viewer • Updated Feb 28 • 10.9k • 80 • 1
umarigan/openhermespreference_tr

Viewer • Updated Apr 24 • 989k • 42 • 3
amztheory/toxic-aira-dataset

Viewer • Updated Apr 30 • 8.29k • 34 • 1
RLHFlow/HH-RLHF-Helpful-standard

Viewer • Updated Apr 27 • 115k • 106 • 1
trl-internal-testing/descriptiveness-sentiment-trl-style

Viewer • Updated Apr 9 • 10.9k • 5.05k • 1
We-Want-GPU/yi-ko-DPO-noprompt-dataset

Viewer • Updated Dec 6, 2023 • 3.83k • 35 • 2
hendrydong/preference_700K

Viewer • Updated Sep 28 • 700k • 556 • 9
insub/imdb_prefix20_forDPO_gpt2-large-imdb-FT_siebert_sentiment-roberta-large-english

Viewer • Updated Oct 22, 2023 • 50k • 69 • 2
FredrikBL/Legal-Snigel-DPO

Viewer • Updated May 27 • 2k • 55 • 1
lightblue/mitsu_full_borda

Viewer • Updated May 30 • 2.85k • 42 • 2
MaziyarPanahi/truthy-dpo-v0.1-axolotl

Viewer • Updated May 2 • 1.02k • 55 • 7
RLHFlow/Orca-distibalel-standard

Viewer • Updated Apr 28 • 6.93k • 61 • 1
RLHFlow/prm80k-phase2

Viewer • Updated Apr 28 • 79.5k • 35 • 4
pythainlp/oasst2_dpo_pairs_enth

Viewer • Updated Jun 16 • 5.28k • 48 • 1

Upvote

Collection guide
Browse collections