altomek commited on
Commit
996d58e
1 Parent(s): d8ae4d1

quants upload

Browse files

Mahou-1.5-mistral-nemo-12B Q4 Q8 quant

.gitattributes CHANGED
@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Mahou-1.5-mistral-nemo-12B-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Mahou-1.5-mistral-nemo-12B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Mahou-1.5-mistral-nemo-12B-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Mahou-1.5-mistral-nemo-12B-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Mahou-1.5-mistral-nemo-12B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Mahou-1.5-mistral-nemo-12B.imatrix filter=lfs diff=lfs merge=lfs -text
Mahou-1.5-mistral-nemo-12B-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33629552f38ed6e634d3fe8e45bb5a671e47f83d80322d25d9cdd22dda41dc07
3
+ size 7097914848
Mahou-1.5-mistral-nemo-12B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:366037c5210d9ff0e955744453ca93c55d2c65b5d1179b5adc3d4142da7e886c
3
+ size 7094638048
Mahou-1.5-mistral-nemo-12B-Q4_0_4_4.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1da60e14c9eff2f668565c5e4e555a445a8e6a16985c80e850cb27ae16da0826
3
+ size 7071700448
Mahou-1.5-mistral-nemo-12B-Q4_1.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e23c4b07c3c381f16e0dd03ec96218f99ab21c369b801318859056d0a9847af0
3
+ size 7795217888
Mahou-1.5-mistral-nemo-12B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1a99a0e9589d38dc2e8ac37ba24a90c8c661c251286c56fccfd0eb70656edb7
3
+ size 13022369248
Mahou-1.5-mistral-nemo-12B.imatrix ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dd97c48105b0bb255d99bdc35762a54270d24095ed48c18f6a08ba7ce46edb98
3
+ size 7054410
README.md ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ base_model: flammenai/Mahou-1.5-mistral-nemo-12B
5
+ datasets: flammenai/MahouMix-v1
6
+ inference: false
7
+ ---
8
+
9
+ # Mahou-1.5-mistral-nemo-12B
10
+
11
+ Selected GGUF quants of https://huggingface.co/flammenai/Mahou-1.5-mistral-nemo-12B
README.org.md ADDED
@@ -0,0 +1,156 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ base_model:
5
+ - flammenai/Flammades-Mistral-Nemo-12B
6
+ datasets:
7
+ - flammenai/MahouMix-v1
8
+ model-index:
9
+ - name: Mahou-1.5-mistral-nemo-12B
10
+ results:
11
+ - task:
12
+ type: text-generation
13
+ name: Text Generation
14
+ dataset:
15
+ name: IFEval (0-Shot)
16
+ type: HuggingFaceH4/ifeval
17
+ args:
18
+ num_few_shot: 0
19
+ metrics:
20
+ - type: inst_level_strict_acc and prompt_level_strict_acc
21
+ value: 67.51
22
+ name: strict accuracy
23
+ source:
24
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=flammenai/Mahou-1.5-mistral-nemo-12B
25
+ name: Open LLM Leaderboard
26
+ - task:
27
+ type: text-generation
28
+ name: Text Generation
29
+ dataset:
30
+ name: BBH (3-Shot)
31
+ type: BBH
32
+ args:
33
+ num_few_shot: 3
34
+ metrics:
35
+ - type: acc_norm
36
+ value: 36.26
37
+ name: normalized accuracy
38
+ source:
39
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=flammenai/Mahou-1.5-mistral-nemo-12B
40
+ name: Open LLM Leaderboard
41
+ - task:
42
+ type: text-generation
43
+ name: Text Generation
44
+ dataset:
45
+ name: MATH Lvl 5 (4-Shot)
46
+ type: hendrycks/competition_math
47
+ args:
48
+ num_few_shot: 4
49
+ metrics:
50
+ - type: exact_match
51
+ value: 5.06
52
+ name: exact match
53
+ source:
54
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=flammenai/Mahou-1.5-mistral-nemo-12B
55
+ name: Open LLM Leaderboard
56
+ - task:
57
+ type: text-generation
58
+ name: Text Generation
59
+ dataset:
60
+ name: GPQA (0-shot)
61
+ type: Idavidrein/gpqa
62
+ args:
63
+ num_few_shot: 0
64
+ metrics:
65
+ - type: acc_norm
66
+ value: 3.47
67
+ name: acc_norm
68
+ source:
69
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=flammenai/Mahou-1.5-mistral-nemo-12B
70
+ name: Open LLM Leaderboard
71
+ - task:
72
+ type: text-generation
73
+ name: Text Generation
74
+ dataset:
75
+ name: MuSR (0-shot)
76
+ type: TAUR-Lab/MuSR
77
+ args:
78
+ num_few_shot: 0
79
+ metrics:
80
+ - type: acc_norm
81
+ value: 16.47
82
+ name: acc_norm
83
+ source:
84
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=flammenai/Mahou-1.5-mistral-nemo-12B
85
+ name: Open LLM Leaderboard
86
+ - task:
87
+ type: text-generation
88
+ name: Text Generation
89
+ dataset:
90
+ name: MMLU-PRO (5-shot)
91
+ type: TIGER-Lab/MMLU-Pro
92
+ config: main
93
+ split: test
94
+ args:
95
+ num_few_shot: 5
96
+ metrics:
97
+ - type: acc
98
+ value: 28.91
99
+ name: accuracy
100
+ source:
101
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=flammenai/Mahou-1.5-mistral-nemo-12B
102
+ name: Open LLM Leaderboard
103
+ ---
104
+ ![image/png](https://huggingface.co/flammenai/Mahou-1.0-mistral-7B/resolve/main/mahou1.png)
105
+
106
+ # Mahou-1.5-mistral-nemo-12B
107
+
108
+ Mahou is designed to provide short messages in a conversational context. It is capable of casual conversation and character roleplay.
109
+
110
+ ### Chat Format
111
+
112
+ This model has been trained to use ChatML format.
113
+
114
+ ```
115
+ <|im_start|>system
116
+ {{system}}<|im_end|>
117
+ <|im_start|>{{char}}
118
+ {{message}}<|im_end|>
119
+ <|im_start|>{{user}}
120
+ {{message}}<|im_end|>
121
+ ```
122
+
123
+ ### Roleplay Format
124
+
125
+ - Speech without quotes.
126
+ - Actions in `*asterisks*`
127
+
128
+ ```
129
+ *leans against wall cooly* so like, i just casted a super strong spell at magician academy today, not gonna lie, felt badass.
130
+ ```
131
+
132
+ ### SillyTavern Settings
133
+
134
+ 1. Use ChatML for the Context Template.
135
+ 2. Enable Instruct Mode.
136
+ 3. Use the [Mahou ChatML Instruct preset](https://huggingface.co/datasets/flammenai/Mahou-ST-ChatML-Instruct/raw/main/Mahou.json).
137
+ 4. Use the [Mahou Sampler preset](https://huggingface.co/datasets/flammenai/Mahou-ST-Sampler-Preset/raw/main/Mahou.json).
138
+
139
+ ### Method
140
+
141
+ [ORPO finetuned](https://mlabonne.github.io/blog/posts/2024-04-19_Fine_tune_Llama_3_with_ORPO.html) with 4x H100 for 3 epochs.
142
+
143
+
144
+ ### [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
145
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_flammenai__Mahou-1.5-mistral-nemo-12B)
146
+
147
+ | Metric |Value|
148
+ |-------------------|----:|
149
+ |Avg. |26.28|
150
+ |IFEval (0-Shot) |67.51|
151
+ |BBH (3-Shot) |36.26|
152
+ |MATH Lvl 5 (4-Shot)| 5.06|
153
+ |GPQA (0-shot) | 3.47|
154
+ |MuSR (0-shot) |16.47|
155
+ |MMLU-PRO (5-shot) |28.91|
156
+