mlabonne commited on
Commit
2e67100
β€’
1 Parent(s): 9c9254f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -16,6 +16,8 @@ language:
16
 
17
  # πŸ‘‘ NeuralMonarch-7B
18
 
 
 
19
  NeuralMonarch-7B is a DPO fine-tuned of [mlabonne/Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B/) using the [jondurbin/truthy-dpo-v0.1](https://huggingface.co/datasets/jondurbin/truthy-dpo-v0.1) and [argilla/distilabel-intel-orca-dpo-pairs](https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs) preference datasets.
20
 
21
  It is based on a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
@@ -27,13 +29,13 @@ Special thanks to [Jon Durbin](https://huggingface.co/jondurbin), [Intel](https:
27
 
28
  ## πŸ” Applications
29
 
30
- This model uses a context window of 8k. It is compatible with different templates, like chatml, Llama's and Mistral's chat templates.
31
 
32
  Compared to other 7B models, it displays good performance in instruction following and reasoning tasks. It can also be used for RP and storytelling.
33
 
34
  ## ⚑ Quantized models
35
 
36
- * **GGUF**: TBD
37
 
38
  ## πŸ† Evaluation
39
 
@@ -41,6 +43,7 @@ The evaluation was performed using [LLM AutoEval](https://github.com/mlabonne/ll
41
 
42
  | Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
43
  |---|---:|---:|---:|---:|---:|
 
44
  | [Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B) [πŸ“„](https://gist.github.com/mlabonne/0b8d057c5ece41e0290580a108c7a093) | 62.68 | 45.48 | 77.07 | 78.04 | 50.14 |
45
  | [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) [πŸ“„](https://gist.github.com/mlabonne/88b21dd9698ffed75d6163ebdc2f6cc8) | 52.42 | 42.75 | 72.99 | 52.99 | 40.94 |
46
  | [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B) [πŸ“„](https://gist.github.com/mlabonne/14687f1eb3425b166db511f31f8e66f6) | 53.51 | 43.67 | 73.24 | 55.37 | 41.76 |
 
16
 
17
  # πŸ‘‘ NeuralMonarch-7B
18
 
19
+ **Update 14/02/24: NeuralMonarch-7B is the new best-performing 7B model on Nous' benchmark suite! πŸŽ‰**
20
+
21
  NeuralMonarch-7B is a DPO fine-tuned of [mlabonne/Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B/) using the [jondurbin/truthy-dpo-v0.1](https://huggingface.co/datasets/jondurbin/truthy-dpo-v0.1) and [argilla/distilabel-intel-orca-dpo-pairs](https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs) preference datasets.
22
 
23
  It is based on a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
 
29
 
30
  ## πŸ” Applications
31
 
32
+ This model uses a context window of 8k. It is compatible with the following chat templates (tested with LM Studio): Alpaca, ChatML, and Mistral Instruct.
33
 
34
  Compared to other 7B models, it displays good performance in instruction following and reasoning tasks. It can also be used for RP and storytelling.
35
 
36
  ## ⚑ Quantized models
37
 
38
+ * **GGUF**: https://huggingface.co/mlabonne/NeuralMonarch-7B-GGUF
39
 
40
  ## πŸ† Evaluation
41
 
 
43
 
44
  | Model | Average | AGIEval | GPT4All | TruthfulQA | Bigbench |
45
  |---|---:|---:|---:|---:|---:|
46
+ | [**NeuralMonarch-7B**](https://huggingface.co/mlabonne/NeuralMonarch-7B) [πŸ“„](https://gist.github.com/mlabonne/64050c96c6aa261a8f5b403190c8dee4) | **62.73** | **45.31** | **76.99** | **78.35** | **50.28** |
47
  | [Monarch-7B](https://huggingface.co/mlabonne/Monarch-7B) [πŸ“„](https://gist.github.com/mlabonne/0b8d057c5ece41e0290580a108c7a093) | 62.68 | 45.48 | 77.07 | 78.04 | 50.14 |
48
  | [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) [πŸ“„](https://gist.github.com/mlabonne/88b21dd9698ffed75d6163ebdc2f6cc8) | 52.42 | 42.75 | 72.99 | 52.99 | 40.94 |
49
  | [mlabonne/NeuralHermes-2.5-Mistral-7B](https://huggingface.co/mlabonne/NeuralHermes-2.5-Mistral-7B) [πŸ“„](https://gist.github.com/mlabonne/14687f1eb3425b166db511f31f8e66f6) | 53.51 | 43.67 | 73.24 | 55.37 | 41.76 |