---
language:
- en
- fr
- de
- es
- it
- pt
- ru
- zh
- ja
license: apache-2.0
---
Official [AQLM](https://arxiv.org/abs/2401.06118) quantization of [mistralai/Mistral-Nemo-Instruct-2407
](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) finetuned with [PV-Tuning](https://arxiv.org/abs/2405.14852).

For this quantization, we used 1 codebook of 16 bits and groupsize of 8.

Results: 
| Model      | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | PiQA | Winogrande | Model size, Gb |
|------|------|-------|------|------|------|------|------|------|
|mistralai/Mistral-Nemo-Instruct-2407| None | 0.6819 | 0.5606 | 0.8241 | 0.6332 | 0.8090	| 0.7498 | 24.5 |
|  | 1x16g8 | 0.6071 | 0.5017 | 0.7942 | 0.5930 | 0.7987	| 0.7356 | 5.8 |

**Note**

We used `lm-eval=0.4.0` for evaluation.