Update README.md

README.md

@@ -5,8 +5,42 @@ datasets:
 library_name: adapter-transformers
 ---
 
-I was just was curious to see what will happen when you use DPO for a merge of non DPO optimized model (OpenHermes) and an already DPO optimized model (neural-chat).
 
-The dataset was quality over quantity roughly ~3000 samples but they were high quality (aqccording to the datasets chosen_score).
+<img src="https://cdn-uploads.huggingface.co/production/uploads/63ae02ff20176b2d21669dd6/3dKVj-q2MXSc5jfXiVxzC.jpeg" width="500" alt="Description of the image">
 
-I decided to go with the OpenHermes chat template which is also integrated into this models tokenizer.
+# Evangelion-7B
+
+I was just curious to see if something special might happen if one uses:
+$$
+\text{{Evangelion}} = \text{{high-quality DPO dataset}} + \text{{merge of DPO optimized model and non-DPO optimized model}}
+$$
+
+The underlying model that I used was `/Weyaxi/OpenHermes-2.5-neural-chat-v3-3-Slerp`.
+
+
+
+
+# Dataset
+Dataset: `/argilla/distilabel-intel-orca-dpo-pairs`
+
+The dataset was quality over quantity roughly ~3000 samples but they were high quality (aqccording to the chosen_score).
+The following filters were applied to the original dataset:
+```python
+dataset = dataset.filter(
+    lambda r:
+        r["status"] != "tie" and
+        r["chosen_score"] >= 8 and
+        not r["in_gsm8k_train"]
+)
+```
+
+# Chat Template
+I decided to go with the ChatML template which I also integrated into this models tokenizer.
+```
+<|im_start|>system
+{system}<|im_end|>
+<|im_start|>user
+{user}<|im_end|>
+<|im_start|>assistant
+{asistant}<|im_end|>
+```