File size: 801 Bytes
51ccd37 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
---
language:
- en
- fr
- de
- es
- it
- pt
- ru
- zh
- ja
license: apache-2.0
---
Official [AQLM](https://arxiv.org/abs/2401.06118) quantization of [mistralai/Mistral-Nemo-Instruct-2407
](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) finetuned with [PV-Tuning](https://arxiv.org/abs/2405.14852).
For this quantization, we used 1 codebook of 16 bits and groupsize of 8.
Results:
| Model | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | PiQA | Winogrande | Model size, Gb |
|------|------|-------|------|------|------|------|------|------|
|mistralai/Mistral-Nemo-Instruct-2407| None | 0.6819 | 0.5606 | 0.8241 | 0.6332 | 0.8090 | 0.7498 | 24.5 |
| | 1x16g8 | 0.6071 | 0.5017 | 0.7942 | 0.5930 | 0.7987 | 0.7356 | 5.4 |
**Note**
We used `lm-eval=0.4.0` for evaluation. |