|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
- es |
|
- de |
|
- fr |
|
- pt |
|
- it |
|
- ru |
|
- ja |
|
- zh |
|
- ko |
|
base_model: |
|
- mistralai/Mistral-Large-Instruct-2411 |
|
tags: |
|
- conversational |
|
- mlx |
|
--- |
|
# Model Card for Mistral-Large-Instruct-2411-MLX |
|
|
|
This is a 2bit quantization of the Mistral Large Instruct 2411 model for MLX (Apple silicon). It was created using the mlx-lm library with the following CLI command: |
|
mlx_lm.convert \ |
|
--hf-path /path/to/your/fp16/model \ |
|
-q \ |
|
--q-bits 2 \ |
|
--q-group-size 32 |
|
|
|
## Quantized Versions |
|
|
|
- [2-bit Quantization (Q2)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q2-MLX) |
|
- [4-bit Quantization (Q4)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q4-MLX) |
|
|
|
Each version is optimized for specific memory and performance trade-offs. |
|
|
|
## Original Model |
|
|
|
The original Mistral-Large-Instruct-2411 model is available [here](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411). Mistral model usage is governed by the [Mistral Research License](https://mistral.ai/licenses/MRL-0.1.md). |
|
|
|
## License |
|
|
|
This model family is governed by the [Mistral Research License](https://mistral.ai/licenses/MRL-0.1.md). Please review the license terms before use. |
|
|
|
## Table of Contents |
|
|
|
- [Model Details](#model-details) |
|
- [Model Description](#model-description) |
|
- [Uses](#uses) |
|
- [Direct Use](#direct-use) |
|
- [Out-of-Scope Use](#out-of-scope-use) |
|
- [Bias, Risks, and Limitations](#bias-risks-and-limitations) |
|
- [Recommendations](#recommendations) |
|
- [Technical Specifications](#technical-specifications) |
|
- [How to Get Started](#how-to-get-started) |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
The Mistral-Large-Instruct-2411-MLX family includes quantized versions of the Mistral Large Instruct 2411 model, optimized for deployment on MLX (Apple Silicon). The quantization reduces memory usage and inference latency, enabling efficient deployment on resource-constrained systems. |
|
|
|
- **Developed by:** Mistral AI |
|
- **Model type:** Large language model |
|
- **Language(s):** English, French, German, Spanish, Italian, Portuguese, Chinese, Japanese, Russian, Korean |
|
- **Quantization levels:** 2-bit (Q2), 4-bit (Q4) |
|
|
|
## Technical Specifications |
|
|
|
- **Parent Model:** [Mistral-Large-Instruct-2411](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411) |
|
- **Quantization:** 2-bit (Q2), 4-bit (Q4) |
|
- **Framework:** MLX (`mlx-lm` library) |
|
|
|
## How to Get Started |
|
|
|
Visit the individual quantized repositories for details and usage instructions: |
|
|
|
- [2-bit Quantization (Q2)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q2-MLX) |
|
- [4-bit Quantization (Q4)](https://huggingface.co/zachlandes/Mistral-Large-Instruct-2411-Q4-MLX) |
|
|
|
## Model Card Contact |
|
|
|
For inquiries, contact [Zach Landes](https://www.linkedin.com/in/zachlandes/). |
