Update README.md
Browse files
README.md
CHANGED
|
@@ -15,7 +15,6 @@ tags:
|
|
| 15 |
|
| 16 |
## VAGO solutions SauerkrautLM-Qwen-32b
|
| 17 |
Introducing **SauerkrautLM-Qwen-32b** – our Sauerkraut version of the powerful [Qwen/Qwen1.5-32B](https://huggingface.co/Qwen/Qwen1.5-32B)!
|
| 18 |
-
**It is an early stage finetuned model and should be used with caution!**
|
| 19 |
|
| 20 |
The model **SauerkrautLM-Qwen-32b** is a **joint effort** between **VAGO solutions** and **Hyperspace.ai.**
|
| 21 |
|
|
@@ -49,7 +48,7 @@ The model **SauerkrautLM-Qwen-32b** is a **joint effort** between **VAGO solutio
|
|
| 49 |
|
| 50 |
### Training procedure:
|
| 51 |
We trained this model for 2 epochs on 160k data samples with SFT.
|
| 52 |
-
Afterwards we applied DPO with 110k data.
|
| 53 |
|
| 54 |
**We teached German language skills on this model.** As far as we know, it is the first Qwen 32B model with bilingual skills in German and English. Nevertheless, formulations may occur that are not entirely correct (still work in progress).
|
| 55 |
|
|
|
|
| 15 |
|
| 16 |
## VAGO solutions SauerkrautLM-Qwen-32b
|
| 17 |
Introducing **SauerkrautLM-Qwen-32b** – our Sauerkraut version of the powerful [Qwen/Qwen1.5-32B](https://huggingface.co/Qwen/Qwen1.5-32B)!
|
|
|
|
| 18 |
|
| 19 |
The model **SauerkrautLM-Qwen-32b** is a **joint effort** between **VAGO solutions** and **Hyperspace.ai.**
|
| 20 |
|
|
|
|
| 48 |
|
| 49 |
### Training procedure:
|
| 50 |
We trained this model for 2 epochs on 160k data samples with SFT.
|
| 51 |
+
Afterwards we applied DPO for 1 epoch with 110k data.
|
| 52 |
|
| 53 |
**We teached German language skills on this model.** As far as we know, it is the first Qwen 32B model with bilingual skills in German and English. Nevertheless, formulations may occur that are not entirely correct (still work in progress).
|
| 54 |
|