Commit
•
19b67c5
1
Parent(s):
1029dbb
Update README.md
Browse files
README.md
CHANGED
@@ -6,4 +6,36 @@ datasets:
|
|
6 |
language:
|
7 |
- en
|
8 |
---
|
9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
language:
|
7 |
- en
|
8 |
---
|
9 |
+
# Zephyr-7B-DICE-Iter1
|
10 |
+
|
11 |
+
This model was developed using [Bootstrapping Language Models with DPO Implicit Rewards](https://arxiv.org/abs/2406.09760) (DICE) at iteration 1, based on the [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) as the starting point.
|
12 |
+
|
13 |
+
## Links to Other Models
|
14 |
+
- [Zephyr-7B-DICE-Iter1](https://huggingface.co/sail/Zephyr-7B-DICE-Iter1)
|
15 |
+
- [Zephyr-7B-DICE-Iter2](https://huggingface.co/sail/Zephyr-7B-DICE-Iter2)
|
16 |
+
|
17 |
+
## Model Description
|
18 |
+
|
19 |
+
- Model type: A 7B parameter GPT-like model fine-tuned on synthetic datasets.
|
20 |
+
- Language(s) (NLP): Primarily English
|
21 |
+
- License: MIT
|
22 |
+
- Fine-tuned from model: HuggingFaceH4/zephyr-7b-beta
|
23 |
+
|
24 |
+
## [AlpacaEval Leaderboard Evaluation Results](https://tatsu-lab.github.io/alpaca_eval/)
|
25 |
+
|
26 |
+
| Model | LC. Win Rate | Win Rate |
|
27 |
+
|-------------------------------------------|:------------:|:--------:|
|
28 |
+
|[Zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) |12.69 |10.71
|
29 |
+
|[Zephyr-7B-DICE-Iter1](https://huggingface.co/sail/Zephyr-7B-DICE-Iter1) |19.03 |17.67
|
30 |
+
|[Zephyr-7B-DICE-Iter2](https://huggingface.co/sail/Zephyr-7B-DICE-Iter2) |**20.71** |**20.16**
|
31 |
+
|
32 |
+
## Citation
|
33 |
+
|
34 |
+
```bibtex
|
35 |
+
@article{chen2024bootstrapping,
|
36 |
+
title={Bootstrapping Language Models with DPO Implicit Rewards},
|
37 |
+
author={Chen, Changyu and Liu, Zichen and Du, Chao and Pang, Tianyu and Liu, Qian and Sinha, Arunesh and Varakantham, Pradeep and Lin, Min},
|
38 |
+
journal={arXiv preprint arXiv:2406.09760},
|
39 |
+
year={2024}
|
40 |
+
}
|
41 |
+
```
|