Update README.md
Browse files
README.md
CHANGED
@@ -16,6 +16,8 @@ language:
|
|
16 |
|
17 |
# π NeuralMonarch-7B
|
18 |
|
|
|
|
|
19 |
NeuralMonarch-7B is a DPO fine-tuned of [mlabonne/Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B/) using the [jondurbin/truthy-dpo-v0.1](https://huggingface.co/datasets/jondurbin/truthy-dpo-v0.1) and [argilla/distilabel-intel-orca-dpo-pairs](https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs) preference datasets.
|
20 |
|
21 |
It is based on a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
|
@@ -27,13 +29,13 @@ Special thanks to [Jon Durbin](https://huggingface.co/jondurbin), [Intel](https:
|
|
27 |
|
28 |
## π Applications
|
29 |
|
30 |
-
This model uses a context window of 8k. It is compatible with
|
31 |
|
32 |
Compared to other 7B models, it displays good performance in instruction following and reasoning tasks. It can also be used for RP and storytelling.
|
33 |
|
34 |
## β‘ Quantized models
|
35 |
|
36 |
-
* **GGUF**:
|
37 |
|
38 |
## π Evaluation
|
39 |
|
@@ -41,6 +43,7 @@ The evaluation was performed using [LLM AutoEval](https://github.com/mlabonne/ll
|
|
41 |
|
42 |
| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
|
43 |
|---|---:|---:|---:|---:|---:|
|
|
|
44 |
| [Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B) [π](https://gist.github.com/mlabonne/0b8d057c5ece41e0290580a108c7a093) | 62.68 | 45.48 | 77.07 | 78.04 | 50.14 |
|
45 |
| [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) [π](https://gist.github.com/mlabonne/88b21dd9698ffed75d6163ebdc2f6cc8) | 52.42 | 42.75 | 72.99 | 52.99 | 40.94 |
|
46 |
| [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B) [π](https://gist.github.com/mlabonne/14687f1eb3425b166db511f31f8e66f6) | 53.51 | 43.67 | 73.24 | 55.37 | 41.76 |
|
|
|
16 |
|
17 |
# π NeuralMonarch-7B
|
18 |
|
19 |
+
**Update 14/02/24: NeuralMonarch-7B is the new best-performing 7B model on Nous' benchmark suite! π**
|
20 |
+
|
21 |
NeuralMonarch-7B is a DPO fine-tuned of [mlabonne/Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B/) using the [jondurbin/truthy-dpo-v0.1](https://huggingface.co/datasets/jondurbin/truthy-dpo-v0.1) and [argilla/distilabel-intel-orca-dpo-pairs](https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs) preference datasets.
|
22 |
|
23 |
It is based on a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
|
|
|
29 |
|
30 |
## π Applications
|
31 |
|
32 |
+
This model uses a context window of 8k. It is compatible with the following chat templates (tested with LM Studio): Alpaca, ChatML, and Mistral Instruct.
|
33 |
|
34 |
Compared to other 7B models, it displays good performance in instruction following and reasoning tasks. It can also be used for RP and storytelling.
|
35 |
|
36 |
## β‘ Quantized models
|
37 |
|
38 |
+
* **GGUF**: https://huggingface.co/mlabonne/NeuralMonarch-7B-GGUF
|
39 |
|
40 |
## π Evaluation
|
41 |
|
|
|
43 |
|
44 |
| Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
|
45 |
|---|---:|---:|---:|---:|---:|
|
46 |
+
| [**NeuralMonarch-7B**](https://huggingface.co/mlabonne/NeuralMonarch-7B) [π](https://gist.github.com/mlabonne/64050c96c6aa261a8f5b403190c8dee4) | **62.73** | **45.31** | **76.99** | **78.35** | **50.28** |
|
47 |
| [Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B) [π](https://gist.github.com/mlabonne/0b8d057c5ece41e0290580a108c7a093) | 62.68 | 45.48 | 77.07 | 78.04 | 50.14 |
|
48 |
| [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) [π](https://gist.github.com/mlabonne/88b21dd9698ffed75d6163ebdc2f6cc8) | 52.42 | 42.75 | 72.99 | 52.99 | 40.94 |
|
49 |
| [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B) [π](https://gist.github.com/mlabonne/14687f1eb3425b166db511f31f8e66f6) | 53.51 | 43.67 | 73.24 | 55.37 | 41.76 |
|