---
license: mit
language:
- en
tags:
- ODIN
- RLHF
- PPO
---

## Model Details
This is an official implementation of ODIN-ppo-L230-7B model, which is a chat assistant trained by fine-tuning LLaMA on Open-Assistant dataset via PPO. 
The L230 means the output length in LIMA test set is ~230. ODIN is the reward model for the training.

## Model Description 


<!-- Provide a longer summary of what this model is. -->
- **Developed by:** [Lichang-Chen](https://huggingface.co/Lichang-Chen) and [Chen Zhu](https://scholar.google.com/citations?hl=zh-CN&user=m-om5O8AAAAJ)
- **Model type:** RLHF model.
- **Language(s) (NLP):** English
- **Finetuned from model:** [Vicuna-7b](https://huggingface.co/lmsys/vicuna-7b-v1.5)

### Model Sources

<!-- Provide the basic links for the model. -->

- **Repository:** [ODIN](https://github.com/Lichang-Chen/ODIN)
- **Paper:** [ODIN: Disentangled Reward Mitigates Hacking in RLHF](https://huggingface.co/papers/2402.07319)