BramVanroy
commited on
Commit
•
e3dd55d
1
Parent(s):
a562cf2
Update README.md
Browse files
README.md
CHANGED
@@ -4,7 +4,6 @@ language:
|
|
4 |
license: cc-by-nc-4.0
|
5 |
tags:
|
6 |
- alignment-handbook
|
7 |
-
- generated_from_trainer
|
8 |
- trl
|
9 |
- dpo
|
10 |
- geitje
|
@@ -37,6 +36,21 @@ This model is a fine-tuned version of [BramVanroy/GEITje-7B-ultra-sft](https://h
|
|
37 |
|
38 |
This is a Dutch instruction/chat model ultimately based on Mistral and aligned with AI feedback via DPO. It is a DPO continuation of the SFT trained [BramVanroy/GEITje-7B-ultra-sft](https://huggingface.co/BramVanroy/GEITje-7B-ultra-sft), which in turn is based on [Rijgersberg/GEITje-7B](https://huggingface.co/Rijgersberg/GEITje-7B), which in turn is based on Mistral 7B and further pretrained on Dutch data. In (rather naive) [benchmarks](https://huggingface.co/spaces/BramVanroy/open_dutch_llm_leaderboard) it outperforms all the original GEITje models on average (but barely). However, note that these benchmarks should be taken with a massive grain of salt (see the disclaimer below the benchmarks on that page). The best evaluation is to try the models and see for yourself.
|
39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
|
41 |
## Usage
|
42 |
|
@@ -201,5 +215,4 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
|
|
201 |
|MATH Lvl 5 (4-Shot)| 0.91|
|
202 |
|GPQA (0-shot) | 1.68|
|
203 |
|MuSR (0-shot) | 1.52|
|
204 |
-
|MMLU-PRO (5-shot) |11.24|
|
205 |
-
|
|
|
4 |
license: cc-by-nc-4.0
|
5 |
tags:
|
6 |
- alignment-handbook
|
|
|
7 |
- trl
|
8 |
- dpo
|
9 |
- geitje
|
|
|
36 |
|
37 |
This is a Dutch instruction/chat model ultimately based on Mistral and aligned with AI feedback via DPO. It is a DPO continuation of the SFT trained [BramVanroy/GEITje-7B-ultra-sft](https://huggingface.co/BramVanroy/GEITje-7B-ultra-sft), which in turn is based on [Rijgersberg/GEITje-7B](https://huggingface.co/Rijgersberg/GEITje-7B), which in turn is based on Mistral 7B and further pretrained on Dutch data. In (rather naive) [benchmarks](https://huggingface.co/spaces/BramVanroy/open_dutch_llm_leaderboard) it outperforms all the original GEITje models on average (but barely). However, note that these benchmarks should be taken with a massive grain of salt (see the disclaimer below the benchmarks on that page). The best evaluation is to try the models and see for yourself.
|
38 |
|
39 |
+
## Citation
|
40 |
+
|
41 |
+
If you use GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:
|
42 |
+
|
43 |
+
```bibtex
|
44 |
+
@misc{vanroy2024geitje7bultraconversational,
|
45 |
+
title={GEITje 7B Ultra: A Conversational Model for Dutch},
|
46 |
+
author={Bram Vanroy},
|
47 |
+
year={2024},
|
48 |
+
eprint={2412.04092},
|
49 |
+
archivePrefix={arXiv},
|
50 |
+
primaryClass={cs.CL},
|
51 |
+
url={https://arxiv.org/abs/2412.04092},
|
52 |
+
}
|
53 |
+
```
|
54 |
|
55 |
## Usage
|
56 |
|
|
|
215 |
|MATH Lvl 5 (4-Shot)| 0.91|
|
216 |
|GPQA (0-shot) | 1.68|
|
217 |
|MuSR (0-shot) | 1.52|
|
218 |
+
|MMLU-PRO (5-shot) |11.24|
|
|