Probably DPO datasets

davanstrien 's Collections

synthetic-data-generation-demos

sentence-transformers-from-synthetic-data

Synthetic (text) Dataset Generation

haiku

Historic language modeling

Climate

Sourced from Wikimedia

Legal Named Entity Recognition

Top 10% instruction tuning datasets

Top 10 Instruction Tuning Datasets copy

Metadata-generation

MOE papers to read

German Text Embedding Clustering Benchmark datasets

cosmochat-reading-list

datasets-tldr-project

Probably DPO datasets

Probably Alpaca Style Datasets

Direct Preference Optimization Datasets

Image Preference Optimization Datasets

query-to-hub-datasets-viewer-project

updated Jun 26

A collection of datasets that probably support DPO

Upvote

HuggingFaceH4/ultrafeedback_binarized

Viewer • Updated Oct 16 • 187k • 6.29k • 244
mlabonne/orpo-dpo-mix-40k

Viewer • Updated Oct 17 • 44.2k • 1.62k • 248
argilla/OpenHermesPreferences

Viewer • Updated Mar 1 • 989k • 2.14k • 199
argilla/distilabel-capybara-dpo-7k-binarized

Viewer • Updated Jul 16 • 7.56k • 961 • 175
argilla/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 5 • 12.9k • 939 • 167
argilla/dpo-mix-7k

Viewer • Updated Jul 16 • 7.5k • 305 • 155
argilla/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 60.9k • 8.31k • 125
Dahoas/rm-static

Viewer • Updated Mar 6, 2023 • 81.4k • 8.68k • 111
openbmb/UltraInteract_pair

Viewer • Updated Apr 5 • 220k • 691 • 104
openbmb/RLAIF-V-Dataset

Viewer • Updated 19 days ago • 33.8k • 2.08k • 126
hiyouga/DPO-En-Zh-20k

Viewer • Updated Jun 7 • 20k • 273 • 88
Dahoas/full-hh-rlhf

Viewer • Updated Feb 23, 2023 • 125k • 1.28k • 72
argilla/OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Feb 10 • 9.79k • 68 • 64
yitingxie/rlhf-reward-datasets

Viewer • Updated Jan 1, 2023 • 81.4k • 410 • 61
allenai/ultrafeedback_binarized_cleaned

Viewer • Updated Dec 1, 2023 • 186k • 929 • 68
allenai/reward-bench

Viewer • Updated Sep 9 • 8.11k • 7.15k • 75
CarperAI/openai_summarize_comparisons

Viewer • Updated Feb 27, 2023 • 260k • 1.76k • 40
beyond/rlhf-reward-single-round-trans_chinese

Viewer • Updated Jul 5, 2023 • 24.9k • 94 • 37
argilla/Capybara-Preferences

Viewer • Updated May 9 • 15.4k • 341 • 39
snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset

Viewer • Updated Jan 23 • 62.7k • 121 • 42
tasksource/oasst1_pairwise_rlhf_reward

Viewer • Updated Jul 4, 2023 • 18.9k • 73 • 42
wenbopan/Chinese-dpo-pairs

Viewer • Updated Apr 2 • 10.7k • 91 • 39
maywell/ko_Ultrafeedback_binarized

Viewer • Updated Nov 9, 2023 • 62k • 60 • 28
abacusai/MetaMath_DPO_FewShot

Viewer • Updated Feb 26 • 395k • 129 • 25
HuggingFaceH4/orca_dpo_pairs

Viewer • Updated Apr 14 • 12.9k • 174 • 25
Dahoas/static-hh

Viewer • Updated Mar 6, 2023 • 101k • 71 • 20
mlabonne/chatml-OpenHermes2.5-dpo-binarized-alpha

Viewer • Updated Mar 21 • 9.2k • 48 • 19
HuggingFaceH4/grok-conversation-harmless

Viewer • Updated Feb 2 • 44.8k • 87 • 21
allenai/preference-test-sets

Viewer • Updated Mar 14 • 43.2k • 419 • 19
abacusai/ARC_DPO_FewShot

Viewer • Updated Feb 26 • 4.25k • 67 • 17
cyberagent/chatbot-arena-ja-calm2-7b-chat-experimental

Viewer • Updated Aug 15 • 29.2k • 59 • 16
vicgalle/configurable-system-prompt-multitask

Viewer • Updated Apr 23 • 1.95k • 164 • 19
coseal/CodeUltraFeedback_binarized

Viewer • Updated Mar 18 • 9.5k • 56 • 15
NobodyExistsOnTheInternet/ToxicDPOqa

Viewer • Updated Apr 26 • 6.87k • 47 • 14
multilingual/orca_dpo_pairs

Viewer • Updated Apr 3 • 64.7k • 133 • 15
fblgit/simple-math-DPO

Viewer • Updated Aug 1 • 800k • 184 • 16
HuggingFaceH4/cai-conversation-harmless

Viewer • Updated Feb 2 • 44.8k • 183 • 14
alvarobartt/dpo-mix-7k-simplified

Viewer • Updated Mar 26 • 7.5k • 63 • 11
kyujinpy/orca_math_dpo

Viewer • Updated Apr 12 • 15.3k • 54 • 18
davanstrien/dataset-tldr-preference-dpo

Viewer • Updated Apr 30 • 522 • 49 • 11
trl-internal-testing/hh-rlhf-trl-style

Viewer • Updated Mar 13 • 169k • 530 • 10
abacusai/HellaSwag_DPO_FewShot

Viewer • Updated Feb 26 • 150k • 52 • 8
2A2I/argilla-dpo-mix-7k-arabic

Viewer • Updated Mar 18 • 7.5k • 60 • 9
argilla/Capybara-Preferences-Filtered

Viewer • Updated Apr 20 • 14.8k • 78 • 9
2A2I/Aya-Command.R-DPO

Viewer • Updated May 16 • 14.2k • 62 • 7
kuotient/orca-math-korean-preference

Viewer • Updated Apr 10 • 193k • 60 • 10
M4-ai/prm_dpo_pairs

Viewer • Updated Jul 1 • 93.9k • 58 • 7
zerolink/zsql-postgres-dpo

Viewer • Updated Feb 2 • 259k • 70 • 6
princeton-nlp/llama3-ultrafeedback

Viewer • Updated Jul 18 • 61.8k • 960 • 15
kaist-ai/Multifaceted-Collection-ORPO

Viewer • Updated Jul 1 • 64.6k • 54 • 9
kaitchup/UltraFeedback-prompt-chosen-rejected

Viewer • Updated Oct 23, 2023 • 18k • 73 • 6
alexredna/oasst2_dpo_pairs

Viewer • Updated Jul 17 • 10k • 60 • 6
HuggingFaceH4/grok-conversation-harmless2

Viewer • Updated Jan 19 • 44.8k • 93 • 6
yongtae-jp/orca_dpo_pairs_ja

Viewer • Updated Jan 9 • 12.9k • 79 • 6
kaist-ai/Multifaceted-Collection-DPO

Viewer • Updated Jul 1 • 65.1k • 65 • 8
malhajar/distilabel-intel-orca-dpo-pairs-tr

Viewer • Updated Mar 28 • 9.12k • 44 • 6
efederici/alpaca-vs-alpaca-orpo-dpo

Viewer • Updated May 15 • 49.2k • 115 • 7
maywell/ko_Ultrafeedback_binarized_10k

Viewer • Updated Nov 4, 2023 • 10k • 80 • 5
kira/math-dpo

Viewer • Updated Jul 2 • 2.42k • 75 • 7
mlabonne/distilabel-truthy-dpo-v0.1

Viewer • Updated Feb 12 • 1.02k • 53 • 5
RLHFlow/UltraFeedback-preference-standard

Viewer • Updated Apr 27 • 340k • 85 • 8
BramVanroy/orca_dpo_pairs_dutch

Viewer • Updated Apr 24 • 11k • 76 • 5
argilla/ultrafeedback-multi-binarized-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 158k • 200 • 6
malhajar/orca_dpo_pairs-tr

Viewer • Updated Mar 30 • 10.8k • 130 • 6
lightblue/mitsu_tophalf_borda

Viewer • Updated May 30 • 1.42k • 63 • 5
NickyNicky/neovalle_H4rmony_dpo_translated_English_to_Spanish

Viewer • Updated May 17 • 2.02k • 42 • 4
maywell/ko_hh-rlhf-20k_filtered

Viewer • Updated Nov 4, 2023 • 19.4k • 37 • 4
kaist-ai/Multifaceted-Collection-RM

Viewer • Updated Jun 7 • 64.9k • 41 • 4
HuggingFaceH4/hh-rlhf-h4

Viewer • Updated Nov 29, 2023 • 169k • 59 • 4
orpo-explorers/ultrafeedback_binarized_15k

Viewer • Updated Apr 18 • 15.4k • 34 • 4
We-Want-GPU/Yi-Ko-DPO-Orca-DPO-Pairs

Viewer • Updated Dec 18, 2023 • 12.7k • 94 • 5
zerolink/zsql-sqlite-dpo

Viewer • Updated Feb 2 • 260k • 72 • 3
argilla/ultrafeedback-multi-binarized-quality-preferences-cleaned

Viewer • Updated Dec 11, 2023 • 155k • 45 • 4
SAGI-1/Greetings_DPO_dataset_V1

Viewer • Updated Dec 18, 2023 • 891 • 44 • 3
2A2I/Aya-SambaLingo.Arabic.Chat-DPO

Viewer • Updated May 25 • 14.2k • 50 • 3
AlekseyKorshuk/evol-codealpaca-v1-dpo

Viewer • Updated Jan 25 • 39.9k • 45 • 4
mlabonne/ultrafeedback-binarized-preferences-cleaned

Viewer • Updated Mar 27 • 28k • 49 • 3
kuotient/orca-math-korean-dpo-pairs

Viewer • Updated Apr 5 • 193k • 112 • 9
mii-community/ultrafeedback-preferences-translated-ita

Viewer • Updated Feb 21 • 60.9k • 70 • 3
sqrti/SPA-VL

Viewer • Updated Jul 3 • 101k • 1.1k • 7
HuggingFaceH4/orca_dpo_pairs_no_system_prompt

Viewer • Updated Jan 15 • 12.9k • 39 • 3
mlabonne/distilabel-intel-orca-dpo-pairs

Viewer • Updated Feb 6 • 5.92k • 51 • 3
d0rj/rlhf-reward-datasets-ru

Viewer • Updated Jul 2 • 81.4k • 80 • 4
RLHFlow/HH-RLHF-Harmless-and-RedTeam-standard

Viewer • Updated May 8 • 42.3k • 41 • 2
2A2I/Aya-AceGPT.13B.Chat-DPO

Viewer • Updated May 16 • 13k • 69 • 2
RLHFlow/Helpsteer-preference-standard

Viewer • Updated Apr 27 • 37.1k • 161 • 4
IlyaGusev/saiga_preferences

Viewer • Updated 24 days ago • 30.6k • 161 • 4
RLHFlow/CodeUltraFeedback-standard

Viewer • Updated Apr 27 • 50.2k • 52 • 4
RLHFlow/UltraInteract-filtered-standard

Viewer • Updated Apr 28 • 162k • 53 • 2
RLHFlow/Prometheus2-preference-standard

Viewer • Updated May 5 • 200k • 54 • 2
RLHFlow/PKU-SafeRLHF-30K-standard

Viewer • Updated Apr 29 • 26.9k • 61 • 2
RLHFlow/Argilla-Math-DPO-standard

Viewer • Updated Apr 30 • 2.42k • 42 • 2
droussis/UltraSafety_binarized-orpo-dpo

Viewer • Updated Apr 25 • 2.26k • 49 • 2
We-Want-GPU/Yi-Ko-DPO-dataset

Viewer • Updated Dec 9, 2023 • 4k • 34 • 4
roborovski/glaive-tool-usage-dpo

Viewer • Updated Feb 29 • 42k • 44 • 2
vicgalle/OpenHermesPreferences-roleplay

Viewer • Updated Feb 29 • 3.06k • 50 • 4
GrowingApple/orpo_kor_translated_medical

Viewer • Updated Jun 12 • 10.4k • 43 • 3
reciprocate/ultrafeedback_cleaned_high_dpo

Viewer • Updated Jan 11 • 40k • 34 • 2
mlabonne/distilabel-truthy-dpo-v0.1-filtered

Viewer • Updated Feb 15 • 663 • 42 • 2
xiaodongguaAIGC/awesome-dpo

Viewer • Updated Jul 3 • 302k • 47 • 2
jan-hq/evol_codealpaca_dpo_binarized

Viewer • Updated Feb 20 • 39.9k • 42 • 2
2A2I/Aya-Aya.23.8B-DPO

Viewer • Updated May 29 • 14k • 59 • 2
tasksource/oasst2_pairwise_rlhf_reward

Viewer • Updated Jan 9 • 28.4k • 55 • 2
NickyNicky/function_call_orpo_sft_phi3_chatML_only

Viewer • Updated May 2 • 66k • 47 • 2
HuggingFaceH4/summarize_from_feedback

Viewer • Updated Mar 11 • 353k • 55 • 2
weqweasdas/preference_dataset_mix2

Viewer • Updated Mar 21 • 528k • 51 • 3
NickyNicky/oasst2_orpo_mix_function_call_phi_3_v1

Viewer • Updated May 2 • 120k • 38 • 2
BramVanroy/ultra_feedback_dutch

Viewer • Updated Apr 18 • 53.6k • 70 • 2
orpo-explorers/OpenHermesPreferences-10k

Viewer • Updated Mar 30 • 10k • 41 • 2
orpo-explorers/OpenHermesPreferences-500k

Viewer • Updated Mar 30 • 500k • 45 • 2
zhengr/ultrafeedback_binarized

Viewer • Updated Nov 8, 2023 • 190k • 112 • 2
ibivibiv/cleaned_orca_math_dpo_pairs

Viewer • Updated May 24 • 200k • 54 • 1
NickyNicky/nano_finance_200k_en_es_chatML_gemma_orpo_dpo

Viewer • Updated May 29 • 201k • 69 • 1
Baidicoot/hh-rlhf-golden-harmful

Viewer • Updated May 10 • 7.64k • 62 • 1
Tachi67/rm_data_motivation

Viewer • Updated Jun 4 • 44.8k • 45 • 1
trl-internal-testing/hh-rlhf-helpful-base-trl-style

Viewer • Updated May 2 • 46.2k • 7.08k • 9
Mitsuki-Sakamoto/hh-rlhf-reward-model-deberta-v3-large-v2-helpful-2-original_mix_50_random_seed_2

Viewer • Updated Jun 8 • 46.2k • 39 • 1
flyingfishinwater/ultrafeedback_clean

Viewer • Updated Jul 2 • 175k • 73 • 1
manishiitg/argilla-distilabel-intel-orca-dpo-pairs

Viewer • Updated Jan 29 • 23.6k • 61 • 1
Palash123/dpo_anthropic_hh_rlhf

Viewer • Updated Dec 4, 2023 • 168k • 90 • 1
David-Xu/astronomy-stack-dpo-text

Viewer • Updated Mar 8 • 19.9k • 50 • 1
d0rj/full-hh-rlhf-ru

Viewer • Updated Jul 2 • 125k • 85 • 2
vwxyzjn/summarize_from_feedback_oai_preprocessing_1706381144

Viewer • Updated Jan 27 • 179k • 197 • 2
reciprocate/ultrafeedback_orca_math_cleaned_high_dpo

Viewer • Updated Jan 11 • 48.3k • 43 • 1
HuggingFaceH4/OpenHermes-2.5-preferences-v0-deduped

Viewer • Updated Feb 9 • 802k • 46 • 1
mlabonne/truthy-dpo-v0.1

Viewer • Updated Feb 18 • 1.02k • 45 • 1
macadeliccc/distilabel-neurology-preferences-2k-orca-format

Viewer • Updated Feb 22 • 1.99k • 53 • 1
jan-hq/athirdpath_roleplay_dpo_binarized

Viewer • Updated Jan 9 • 3.43k • 35 • 1
reciprocate/ultrafeedback_orca_math_dpo

Viewer • Updated Jan 8 • 73.8k • 42 • 1
d0rj/synthetic-instruct-gptj-pairwise-ru

Viewer • Updated Jul 2 • 33.1k • 356 • 1
CorticalStack/tak-stack-dpo

Viewer • Updated Feb 28 • 10.9k • 71 • 1
umarigan/openhermespreference_tr

Viewer • Updated Apr 24 • 989k • 51 • 2
amztheory/toxic-aira-dataset

Viewer • Updated Apr 30 • 8.29k • 38 • 1
RLHFlow/HH-RLHF-Helpful-standard

Viewer • Updated Apr 27 • 115k • 81 • 1
trl-internal-testing/descriptiveness-sentiment-trl-style

Viewer • Updated Apr 9 • 10.9k • 5.28k • 1
We-Want-GPU/yi-ko-DPO-noprompt-dataset

Viewer • Updated Dec 6, 2023 • 3.83k • 39 • 2
hendrydong/preference_700K

Viewer • Updated Sep 28 • 700k • 1.03k • 7
insub/imdb_prefix20_forDPO_gpt2-large-imdb-FT_siebert_sentiment-roberta-large-english

Viewer • Updated Oct 22, 2023 • 50k • 58 • 2
FredrikBL/Legal-Snigel-DPO

Viewer • Updated May 27 • 2k • 37 • 1
lightblue/mitsu_full_borda

Viewer • Updated May 30 • 2.85k • 49 • 2
MaziyarPanahi/truthy-dpo-v0.1-axolotl

Viewer • Updated May 2 • 1.02k • 76 • 7
RLHFlow/Orca-distibalel-standard

Viewer • Updated Apr 28 • 6.93k • 46 • 1
RLHFlow/prm80k-phase2

Viewer • Updated Apr 28 • 79.5k • 42 • 4
pythainlp/oasst2_dpo_pairs_enth

Viewer • Updated Jun 16 • 5.28k • 40 • 1

Upvote

Collection guide
Browse collections