Hugging Face's logo

RichardErkhov
/
nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf

GGUF
Model card Files Community
RichardErkhov commited on
Commit
1e93d42
•
1 Parent(s): 377f0a2

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +165 -0
README.md ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ OpenMath-Mistral-7B-v0.1-hf - GGUF
11
+ - Model creator: https://huggingface.co/nvidia/
12
+ - Original model: https://huggingface.co/nvidia/OpenMath-Mistral-7B-v0.1-hf/
13
+
14
+
15
+ | Name | Quant method | Size |
16
+ | ---- | ---- | ---- |
17
+ | [OpenMath-Mistral-7B-v0.1-hf.Q2_K.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q2_K.gguf) | Q2_K | 2.53GB |
18
+ | [OpenMath-Mistral-7B-v0.1-hf.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.IQ3_XS.gguf) | IQ3_XS | 2.81GB |
19
+ | [OpenMath-Mistral-7B-v0.1-hf.IQ3_S.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.IQ3_S.gguf) | IQ3_S | 2.96GB |
20
+ | [OpenMath-Mistral-7B-v0.1-hf.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q3_K_S.gguf) | Q3_K_S | 2.95GB |
21
+ | [OpenMath-Mistral-7B-v0.1-hf.IQ3_M.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.IQ3_M.gguf) | IQ3_M | 3.06GB |
22
+ | [OpenMath-Mistral-7B-v0.1-hf.Q3_K.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q3_K.gguf) | Q3_K | 3.28GB |
23
+ | [OpenMath-Mistral-7B-v0.1-hf.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q3_K_M.gguf) | Q3_K_M | 3.28GB |
24
+ | [OpenMath-Mistral-7B-v0.1-hf.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q3_K_L.gguf) | Q3_K_L | 3.56GB |
25
+ | [OpenMath-Mistral-7B-v0.1-hf.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.IQ4_XS.gguf) | IQ4_XS | 3.67GB |
26
+ | [OpenMath-Mistral-7B-v0.1-hf.Q4_0.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q4_0.gguf) | Q4_0 | 3.83GB |
27
+ | [OpenMath-Mistral-7B-v0.1-hf.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.IQ4_NL.gguf) | IQ4_NL | 3.87GB |
28
+ | [OpenMath-Mistral-7B-v0.1-hf.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q4_K_S.gguf) | Q4_K_S | 3.86GB |
29
+ | [OpenMath-Mistral-7B-v0.1-hf.Q4_K.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q4_K.gguf) | Q4_K | 4.07GB |
30
+ | [OpenMath-Mistral-7B-v0.1-hf.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q4_K_M.gguf) | Q4_K_M | 4.07GB |
31
+ | [OpenMath-Mistral-7B-v0.1-hf.Q4_1.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q4_1.gguf) | Q4_1 | 4.24GB |
32
+ | [OpenMath-Mistral-7B-v0.1-hf.Q5_0.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q5_0.gguf) | Q5_0 | 4.65GB |
33
+ | [OpenMath-Mistral-7B-v0.1-hf.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q5_K_S.gguf) | Q5_K_S | 4.65GB |
34
+ | [OpenMath-Mistral-7B-v0.1-hf.Q5_K.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q5_K.gguf) | Q5_K | 4.78GB |
35
+ | [OpenMath-Mistral-7B-v0.1-hf.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q5_K_M.gguf) | Q5_K_M | 4.78GB |
36
+ | [OpenMath-Mistral-7B-v0.1-hf.Q5_1.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q5_1.gguf) | Q5_1 | 5.07GB |
37
+ | [OpenMath-Mistral-7B-v0.1-hf.Q6_K.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q6_K.gguf) | Q6_K | 5.53GB |
38
+ | [OpenMath-Mistral-7B-v0.1-hf.Q8_0.gguf](https://huggingface.co/RichardErkhov/nvidia_-_OpenMath-Mistral-7B-v0.1-hf-gguf/blob/main/OpenMath-Mistral-7B-v0.1-hf.Q8_0.gguf) | Q8_0 | 7.17GB |
39
+
40
+
41
+
42
+
43
+ Original model description:
44
+ ---
45
+ license: apache-2.0
46
+ base_model:
47
+ - mistralai/Mistral-7B-v0.1
48
+ datasets:
49
+ - nvidia/OpenMathInstruct-1
50
+ language:
51
+ - en
52
+ tags:
53
+ - nvidia
54
+ - code
55
+ - math
56
+ ---
57
+
58
+
59
+ # OpenMath-Mistral-7B-v0.1-hf
60
+
61
+ OpenMath models were designed to solve mathematical problems by integrating text-based reasoning with code blocks
62
+ executed by Python interpreter. The models were trained on [OpenMathInstruct-1](https://huggingface.co/datasets/nvidia/OpenMathInstruct-1),
63
+ a math instruction tuning dataset with 1.8M problem-solution pairs generated using permissively licensed
64
+ [Mixtral-8x7B](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) model.
65
+
66
+ <table border="1">
67
+ <tr>
68
+ <td></td>
69
+ <td colspan="2" style="text-align: center;">greedy</td>
70
+ <td colspan="2" style="text-align: center;">majority@50</td>
71
+ </tr>
72
+ <tr>
73
+ <td style="text-align: center;">model</td>
74
+ <td style="text-align: center;">GSM8K</td>
75
+ <td style="text-align: center;">MATH</td>
76
+ <td style="text-align: center;">GMS8K</td>
77
+ <td style="text-align: center;">MATH</td>
78
+ </tr>
79
+ <tr>
80
+ <td style="text-align: right;">OpenMath-CodeLlama-7B (<a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-7b-Python">nemo</a> | <a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-7b-Python-hf">HF</a>)</td>
81
+ <td style="text-align: center;">75.9</td>
82
+ <td style="text-align: center;">43.6</td>
83
+ <td style="text-align: center;">84.8</td>
84
+ <td style="text-align: center;">55.6</td>
85
+ </tr>
86
+ <tr>
87
+ <td style="text-align: right;">OpenMath-Mistral-7B (<a href="https://huggingface.co/nvidia/OpenMath-Mistral-7B-v0.1">nemo</a> | <a href="https://huggingface.co/nvidia/OpenMath-Mistral-7B-v0.1-hf">HF</a>)</td>
88
+ <td style="text-align: center;">80.2</td>
89
+ <td style="text-align: center;">44.5</td>
90
+ <td style="text-align: center;">86.9</td>
91
+ <td style="text-align: center;">57.2</td>
92
+ </tr>
93
+ <tr>
94
+ <td style="text-align: right;">OpenMath-CodeLlama-13B (<a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-13b-Python">nemo</a> | <a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-13b-Python-hf">HF</a>)</td>
95
+ <td style="text-align: center;">78.8</td>
96
+ <td style="text-align: center;">45.5</td>
97
+ <td style="text-align: center;">86.8</td>
98
+ <td style="text-align: center;">57.6</td>
99
+ </tr>
100
+ <tr>
101
+ <td style="text-align: right;">OpenMath-CodeLlama-34B (<a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-34b-Python">nemo</a> | <a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-34b-Python-hf">HF</a>)</td>
102
+ <td style="text-align: center;">80.7</td>
103
+ <td style="text-align: center;">48.3</td>
104
+ <td style="text-align: center;">88.0</td>
105
+ <td style="text-align: center;">60.2</td>
106
+ </tr>
107
+ <tr>
108
+ <td style="text-align: right;">OpenMath-Llama2-70B (<a href="https://huggingface.co/nvidia/OpenMath-Llama-2-70b">nemo</a> | <a href="https://huggingface.co/nvidia/OpenMath-Llama-2-70b-hf">HF</a>)</td>
109
+ <td style="text-align: center;"><b>84.7</b></td>
110
+ <td style="text-align: center;">46.3</td>
111
+ <td style="text-align: center;">90.1</td>
112
+ <td style="text-align: center;">58.3</td>
113
+ </tr>
114
+ <tr>
115
+ <td style="text-align: right;">OpenMath-CodeLlama-70B (<a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-70b-Python">nemo</a> | <a href="https://huggingface.co/nvidia/OpenMath-CodeLlama-70b-Python-hf">HF</a>)</td>
116
+ <td style="text-align: center;">84.6</td>
117
+ <td style="text-align: center;"><b>50.7</b></td>
118
+ <td style="text-align: center;"><b>90.8</b></td>
119
+ <td style="text-align: center;"><b>60.4</b></td>
120
+ </tr>
121
+ </table>
122
+
123
+ The pipeline we used to produce these models is fully open-sourced!
124
+
125
+ - [Code](https://github.com/Kipok/NeMo-Skills)
126
+ - [Models](https://huggingface.co/collections/nvidia/openmath-65c5619de2ba059be0775014)
127
+ - [Dataset](https://huggingface.co/datasets/nvidia/OpenMathInstruct-1)
128
+
129
+ See our [paper](https://arxiv.org/abs/2402.10176) for more details!
130
+
131
+ # How to use the models?
132
+
133
+ Try to [run inference with our models](https://github.com/Kipok/NeMo-Skills/blob/main/docs/inference.md) with just a few commands!
134
+
135
+ # Reproducing our results
136
+
137
+ We provide [all instructions](https://github.com/Kipok/NeMo-Skills/blob/main/docs/reproducing-results.md) to fully reproduce our results.
138
+
139
+ # Improving other models
140
+
141
+ To improve other models or to learn more about our code, read through the docs below.
142
+
143
+ - [NeMo-Skills Pipeline](https://github.com/Kipok/NeMo-Skills)
144
+ - [Generating synthetic data](https://github.com/Kipok/NeMo-Skills/blob/main/docs/synthetic-data-generation.md)
145
+ - [Finetuning models](https://github.com/Kipok/NeMo-Skills/blob/main/docs/finetuning.md)
146
+ - [Evaluating models](https://github.com/Kipok/NeMo-Skills/blob/main/docs/evaluation.md)
147
+
148
+ In our pipeline we use [NVIDIA NeMo](https://www.nvidia.com/en-us/ai-data-science/generative-ai/nemo-framework/),
149
+ an end-to-end, cloud-native framework to build, customize, and deploy generative AI models anywhere.
150
+ It includes training and inferencing frameworks, guardrailing toolkits, data curation tools, and pretrained models,
151
+ offering enterprises an easy, cost-effective, and fast way to adopt generative AI.
152
+
153
+ # Citation
154
+
155
+ If you find our work useful, please consider citing us!
156
+
157
+ ```bibtex
158
+ @article{toshniwal2024openmath,
159
+ title = {OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset},
160
+ author = {Shubham Toshniwal and Ivan Moshkov and Sean Narenthiran and Daria Gitman and Fei Jia and Igor Gitman},
161
+ year = {2024},
162
+ journal = {arXiv preprint arXiv: Arxiv-2402.10176}
163
+ }
164
+ ```
165
+