zachlandes commited on
Commit
704d1c8
·
1 Parent(s): 07bdf63

Fix readme

Browse files
Files changed (1) hide show
  1. README.md +31 -234
README.md CHANGED
@@ -1,15 +1,5 @@
1
- ---
2
 
3
- ---
4
-
5
-
6
-
7
-
8
-
9
-
10
- # Model Card for Mistral-Large-Instruct-2411-Q2-MLX
11
-
12
- <!-- Provide a quick summary of what the model is/does. [Optional] -->
13
  This is a 2bit quantization of the Mistral Large Instruct 2411 model for MLX (Apple silicon). It was created using the mlx-lm library with the following CLI command:
14
  mlx_lm.convert \
15
  --hf-path /path/to/your/fp16/model \
@@ -17,250 +7,57 @@ mlx_lm.convert \
17
  --q-bits 2 \
18
  --q-group-size 32
19
 
20
- Original model is here:
21
- https://huggingface.co/mistralai/Mistral-Large-Instruct-2411
22
 
23
- Mistral model usage is governed by the Mistral Research License, viewable at the original model page and, at the time of upload, directly at this link:
24
- https://mistral.ai/licenses/MRL-0.1.md
25
 
 
26
 
 
27
 
 
28
 
29
- # Table of Contents
 
 
 
 
30
 
31
- - [Model Card for Mistral-Large-Instruct-2411-Q2-MLX](#model-card-for--model_id-)
32
- - [Table of Contents](#table-of-contents)
33
- - [Table of Contents](#table-of-contents-1)
34
  - [Model Details](#model-details)
35
  - [Model Description](#model-description)
36
  - [Uses](#uses)
37
  - [Direct Use](#direct-use)
38
- - [Downstream Use [Optional]](#downstream-use-optional)
39
  - [Out-of-Scope Use](#out-of-scope-use)
40
  - [Bias, Risks, and Limitations](#bias-risks-and-limitations)
41
  - [Recommendations](#recommendations)
42
- - [Training Details](#training-details)
43
- - [Training Data](#training-data)
44
- - [Training Procedure](#training-procedure)
45
- - [Preprocessing](#preprocessing)
46
- - [Speeds, Sizes, Times](#speeds-sizes-times)
47
- - [Evaluation](#evaluation)
48
- - [Testing Data, Factors & Metrics](#testing-data-factors--metrics)
49
- - [Testing Data](#testing-data)
50
- - [Factors](#factors)
51
- - [Metrics](#metrics)
52
- - [Results](#results)
53
- - [Model Examination](#model-examination)
54
- - [Environmental Impact](#environmental-impact)
55
- - [Technical Specifications [optional]](#technical-specifications-optional)
56
- - [Model Architecture and Objective](#model-architecture-and-objective)
57
- - [Compute Infrastructure](#compute-infrastructure)
58
- - [Hardware](#hardware)
59
- - [Software](#software)
60
- - [Citation](#citation)
61
- - [Glossary [optional]](#glossary-optional)
62
- - [More Information [optional]](#more-information-optional)
63
- - [Model Card Authors [optional]](#model-card-authors-optional)
64
- - [Model Card Contact](#model-card-contact)
65
- - [How to Get Started with the Model](#how-to-get-started-with-the-model)
66
-
67
-
68
- # Model Details
69
-
70
- ## Model Description
71
-
72
- <!-- Provide a longer summary of what this model is/does. -->
73
- This is a 2bit quantization of the Mistral Large Instruct 2411 model for MLX (Apple silicon). It was created using the mlx-lm library with the following CLI command:
74
- mlx_lm.convert \
75
- --hf-path /path/to/your/fp16/model \
76
- -q \
77
- --q-bits 2 \
78
- --q-group-size 32
79
-
80
- Original model is here:
81
- https://huggingface.co/mistralai/Mistral-Large-Instruct-2411
82
-
83
- Mistral model usage is governed by the Mistral Research License, viewable at the original model page and, at the time of upload, directly at this link:
84
- https://mistral.ai/licenses/MRL-0.1.md
85
-
86
- - **Developed by:** More information needed
87
- - **Shared by [Optional]:** More information needed
88
- - **Model type:** Language model
89
- - **Language(s) (NLP):** en, fra, deu, spa, ita, por, zho, jpn, rus, kor
90
- - **License:** apache-2.0
91
- - **Parent Model:** More information needed
92
- - **Resources for more information:** More information needed
93
-
94
-
95
-
96
- # Uses
97
-
98
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
99
-
100
- ## Direct Use
101
-
102
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
103
- <!-- If the user enters content, print that. If not, but they enter a task in the list, use that. If neither, say "more info needed." -->
104
-
105
-
106
-
107
-
108
- ## Downstream Use [Optional]
109
-
110
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
111
- <!-- If the user enters content, print that. If not, but they enter a task in the list, use that. If neither, say "more info needed." -->
112
-
113
-
114
-
115
-
116
- ## Out-of-Scope Use
117
-
118
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
119
- <!-- If the user enters content, print that. If not, but they enter a task in the list, use that. If neither, say "more info needed." -->
120
-
121
-
122
-
123
-
124
- # Bias, Risks, and Limitations
125
-
126
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
127
-
128
- Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
129
-
130
-
131
- ## Recommendations
132
-
133
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
134
-
135
-
136
-
137
-
138
-
139
- # Training Details
140
-
141
- ## Training Data
142
-
143
- <!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
144
-
145
- More information on training data needed
146
-
147
-
148
- ## Training Procedure
149
-
150
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
151
-
152
- ### Preprocessing
153
-
154
- More information needed
155
-
156
- ### Speeds, Sizes, Times
157
-
158
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
159
-
160
- More information needed
161
-
162
- # Evaluation
163
-
164
- <!-- This section describes the evaluation protocols and provides the results. -->
165
-
166
- ## Testing Data, Factors & Metrics
167
-
168
- ### Testing Data
169
-
170
- <!-- This should link to a Data Card if possible. -->
171
-
172
- More information needed
173
-
174
-
175
- ### Factors
176
-
177
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
178
-
179
- More information needed
180
-
181
- ### Metrics
182
-
183
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
184
-
185
- More information needed
186
-
187
- ## Results
188
-
189
- More information needed
190
-
191
- # Model Examination
192
-
193
- More information needed
194
-
195
- # Environmental Impact
196
-
197
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
198
-
199
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
200
-
201
- - **Hardware Type:** More information needed
202
- - **Hours used:** More information needed
203
- - **Cloud Provider:** More information needed
204
- - **Compute Region:** More information needed
205
- - **Carbon Emitted:** More information needed
206
-
207
- # Technical Specifications [optional]
208
-
209
- ## Model Architecture and Objective
210
-
211
- More information needed
212
-
213
- ## Compute Infrastructure
214
-
215
- More information needed
216
-
217
- ### Hardware
218
-
219
- More information needed
220
-
221
- ### Software
222
-
223
- More information needed
224
-
225
- # Citation
226
-
227
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
228
-
229
- **BibTeX:**
230
-
231
- More information needed
232
-
233
- **APA:**
234
-
235
- More information needed
236
-
237
- # Glossary [optional]
238
-
239
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
240
-
241
- More information needed
242
-
243
- # More Information [optional]
244
 
245
- More information needed
246
 
247
- # Model Card Authors [optional]
248
 
249
- <!-- This section provides another layer of transparency and accountability. Whose views is this model card representing? How many voices were included in its construction? Etc. -->
250
 
251
- More information needed
 
 
 
252
 
253
- # Model Card Contact
254
 
255
- More information needed
 
 
256
 
257
- # How to Get Started with the Model
258
 
259
- Use the code below to get started with the model.
260
 
261
- <details>
262
- <summary> Click to expand </summary>
263
 
264
- More information needed
265
 
266
- </details>
 
1
+ # Model Card for Mistral-Large-Instruct-2411-MLX
2
 
 
 
 
 
 
 
 
 
 
 
3
  This is a 2bit quantization of the Mistral Large Instruct 2411 model for MLX (Apple silicon). It was created using the mlx-lm library with the following CLI command:
4
  mlx_lm.convert \
5
  --hf-path /path/to/your/fp16/model \
 
7
  --q-bits 2 \
8
  --q-group-size 32
9
 
10
+ ## Quantized Versions
 
11
 
12
+ - [2-bit Quantization (Q2)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q2-MLX)
13
+ - [4-bit Quantization (Q4)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q4-MLX)
14
 
15
+ Each version is optimized for specific memory and performance trade-offs.
16
 
17
+ ## Original Model
18
 
19
+ The original Mistral-Large-Instruct-2411 model is available [here](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411). Mistral model usage is governed by the [Mistral Research License](https://mistral.ai/licenses/MRL-0.1.md).
20
 
21
+ ## License
22
+
23
+ This model family is governed by the [Mistral Research License](https://mistral.ai/licenses/MRL-0.1.md). Please review the license terms before use.
24
+
25
+ ## Table of Contents
26
 
 
 
 
27
  - [Model Details](#model-details)
28
  - [Model Description](#model-description)
29
  - [Uses](#uses)
30
  - [Direct Use](#direct-use)
 
31
  - [Out-of-Scope Use](#out-of-scope-use)
32
  - [Bias, Risks, and Limitations](#bias-risks-and-limitations)
33
  - [Recommendations](#recommendations)
34
+ - [Technical Specifications](#technical-specifications)
35
+ - [How to Get Started](#how-to-get-started)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
+ ## Model Details
38
 
39
+ ### Model Description
40
 
41
+ The Mistral-Large-Instruct-2411-MLX family includes quantized versions of the Mistral Large Instruct 2411 model, optimized for deployment on MLX (Apple Silicon). The quantization reduces memory usage and inference latency, enabling efficient deployment on resource-constrained systems.
42
 
43
+ - **Developed by:** Mistral AI
44
+ - **Model type:** Large language model
45
+ - **Language(s):** English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Russian, Korean
46
+ - **Quantization levels:** 2-bit (Q2), 4-bit (Q4)
47
 
48
+ ## Technical Specifications
49
 
50
+ - **Parent Model:** [Mistral-Large-Instruct-2411](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411)
51
+ - **Quantization:** 2-bit (Q2), 4-bit (Q4)
52
+ - **Framework:** MLX (`mlx-lm` library)
53
 
54
+ ## How to Get Started
55
 
56
+ Visit the individual quantized repositories for details and usage instructions:
57
 
58
+ - [2-bit Quantization (Q2)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q2-MLX)
59
+ - [4-bit Quantization (Q4)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q4-MLX)
60
 
61
+ ## Model Card Contact
62
 
63
+ For inquiries, contact [Zach Landes](https://www.linkedin.com/in/zachlandes/).