| language: | |
| - en | |
| - fr | |
| - de | |
| - es | |
| - it | |
| - pt | |
| - ru | |
| - zh | |
| - ja | |
| license: apache-2.0 | |
| Official [AQLM](https://arxiv.org/abs/2401.06118) quantization of [mistralai/Mistral-Nemo-Instruct-2407 | |
| ](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) finetuned with [PV-Tuning](https://arxiv.org/abs/2405.14852). | |
| For this quantization, we used 1 codebook of 16 bits and groupsize of 8. | |
| Results: | |
| | Model | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | PiQA | Winogrande | Model size, Gb | | |
| |------|------|-------|------|------|------|------|------|------| | |
| |mistralai/Mistral-Nemo-Instruct-2407| None | 0.6819 | 0.5606 | 0.8241 | 0.6332 | 0.8090 | 0.7498 | 24.5 | | |
| | | 1x16g8 | 0.6071 | 0.5017 | 0.7942 | 0.5930 | 0.7987 | 0.7356 | 5.4 | | |
| **Note** | |
| We used `lm-eval=0.4.0` for evaluation. |