allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm

Text Generation • Updated 13 days ago • 10 • 2

Note Our overall best model, a 13B Tulu 2 model trained using PPO with a 70B reward model trained on UltraFeedback! We also release the value and reward models associated with this model - see the model card for details

Note The datasets used for training PPO, DPO, and reward models in our paper.

Note The prompt sets used during PPO training in our paper. Below, see all our PPO-trained models!

allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm-mixed-prompts

Text Generation • Updated 13 days ago

Note Below is our PPO data ablations.

allenai/tulu-v2.5-ppo-13b-nectar-60k

Text Generation • Updated 13 days ago

Note Below is our DPO data ablations.

allenai/tulu-v2.5-dpo-13b-nectar-60k

Text Generation • Updated 12 days ago • 1 • 1

Note Below are our reward models!

allenai/tulu-v2.5-13b-stackexchange-60k-rm

Text Classification • Updated 12 days ago • 1

Note Below are our value models.