Update README.md

README.md

@@ -74,37 +74,36 @@ However, the average of **71.24** would earn the #2 spot on the HF leaderboard a
 
 We use [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) to run the benchmark tests above, using the same version as the HuggingFace LLM Leaderboard.
 
-<!--
 ### FastEval
 
 | Metric | Value |
 |-----------------------|-------|
-| GSM8K       | 81.2 |
-| Math   | 22.3 |
-| BBH         | 72.9 |
-| MMLU   | 67.9 |
-| **Avg.**                  | **53.3** |
+| GSM8K       | 70.6 |
+| Math   | 17.8 |
+| BBH         | 63.4 |
+| MMLU   | 64.7 |
+| **Avg.**                  | **48.87** |
 
 ### MTBench
 
 ```json
 {
-    "first_turn": 8.45,
-    "second_turn": 7.45,
+    "first_turn": 7.9,
+    "second_turn": 7.0625,
     "categories": {
-        "writing": 9.4,
-        "roleplay": 8.65,
-        "reasoning": 6.85,
-        "math": 5.55,
-        "coding": 4.95,
-        "extraction": 9.15,
-        "stem": 9.225,
-        "humanities": 9.825
+        "writing": 9.55,
+        "roleplay": 8.35,
+        "reasoning": 6.15,
+        "math": 4.7,
+        "coding": 4.8,
+        "extraction": 7.35,
+        "stem": 9.1,
+        "humanities": 9.85
     },
-    "average": 7.95
+    "average": 7.48125
 }
 ```
--->
+
 ## Prompt Format
 
 This model follows the ChatML format:
@@ -132,17 +131,18 @@ If you use `tokenize=True` and `return_tensors="pt"` instead, then you will get
 
 ## Dataset
 
-The dataset curation for DiscoLM 120b followed a "brute force"/"PoC" approach, as one goal was to see whether a 120b model can "absorb" more instruction data than a 70b model.
+The dataset curation for DiscoLM 70b followed a "brute force"/"PoC" approach.
 
-The following datasets were used for training DiscoLM 120b:
+The following datasets were used for training DiscoLM 70b:
 
 * [SlimOrca-Dedup](https://huggingface.co/datasets/Open-Orca/SlimOrca-Dedup)
-* [OpenPlatypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus)
+* [OpenSchnabeltier](https://huggingface.co/datasets/LeoLM/OpenSchnabeltier) translated to DE from [OpenPlatypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus)
 * [OpenHermes](https://huggingface.co/datasets/teknium/openhermes)
 * [MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA)
-* [UltraChat](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k)
+* [UltraChat DE](https://huggingface.co/datasets/bjoernp/ultrachat_de) translated to DE from [UltraChat](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k)
 * [Synthia v.1.3](https://huggingface.co/datasets/migtissera/Synthia-v1.3)
-* [AgentInstruct](https://huggingface.co/datasets/THUDM/AgentInstruct)
+* [German_Songs](https://huggingface.co/datasets/THUDM/AgentInstruct)
+* Capybara Dataset by [Nous Research](https://huggingface.co/NousResearch/)
 
 Many thanks for all dataset providers/curators!
 
@@ -156,10 +156,13 @@ DiscoResearch is an aspiring open research community. Disco should be a place wh
 
 ## Acknowledgements
 
-Disco 120b is a [DiscoResearch](https://huggingface.co/DiscoResearch) project and was trained by [Björn Plüster](https://huggingface.co/bjoernp). [Jan Harries](https://huggingface.co/jphme) helped with technical adivce, logistics and the Model Card and [AutoMeta](https://huggingface.co/Alignment-Lab-AI) also provided helpful technical adivce.
+Disco 70b is a [DiscoResearch](https://huggingface.co/DiscoResearch) project and was trained by [Björn Plüster](https://huggingface.co/bjoernp). [Jan Harries](https://huggingface.co/jphme) helped with technical adivce, logistics and the Model Card. 
+[AutoMeta](https://huggingface.co/Alignment-Lab-AI) also provided helpful technical advice and rounded up his connections to select a set of high-quality datasets.
 The model was trained with compute provided by [HessianAI](https://hessian.ai/) - many thanks in particular to [Patrick Schramowski](https://huggingface.co/PSaiml) for his support. 
 
-We are standing on the shoulders of giants; many thanks in no particular order to [alpindale](https://huggingface.co/alpindale) for Goliath 120b (with important contributions by [Charles Goddard](https://huggingface.co/chargoddard) and [Undi95](https://huggingface.co/Undi95)), [TheBloke](https://huggingface.co/TheBloke) for providing quantized versions, [winglian](https://huggingface.co/winglian) for Axolotl which was used to train the model and the SlimOrca dataset, [garage-bAInd](https://huggingface.co/garage-bAInd), [Teknium](https://huggingface.co/teknium), [Migel Tissera](https://huggingface.co/migtissera), [MetaMath](https://huggingface.co/meta-math) for their great datasets (please contact us if we forgot to mention you here!).
+We are standing on the shoulders of giants; many thanks in no particular order to [Laion](https://laion.ai) for LeoLM 70b
+(especially to [Christoph Schuhmann](https://laion.ai) who got us all connected),
+[TheBloke](https://huggingface.co/TheBloke) for providing quantized versions, [winglian](https://huggingface.co/winglian) for Axolotl which was used to train the model and the SlimOrca dataset, [garage-bAInd](https://huggingface.co/garage-bAInd), [Teknium](https://huggingface.co/teknium), [Migel Tissera](https://huggingface.co/migtissera), [MetaMath](https://huggingface.co/meta-math) for their great datasets (please contact us if we forgot to mention you here!).
 
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)