ISTA-DASLab
/

Meta-Llama-3-70B-AQLM-2Bit-1x16

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

SpiridonSunRotator commited on Apr 29

Commit

00d2f33

•

1 Parent(s): e16ede7

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -13,8 +13,8 @@ Official [AQLM](https://arxiv.org/abs/2401.06118) quantization of [meta-llama/Me
 For this quantization, we used 1 codebook of 16 bits.
-Results (in progress):
 | Model      | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | Winogrande | PiQA | Model size, Gb |
 |------|------|-------|------|------|------|------|------|------|
-|meta-llama/Meta-Llama-3-70B | - | - | 0.6596 | 0.8935 | 0.6921	| 0.8444 | 0.8090 | 141.2 |
 |  | 1x16 | 0.7475| 0.6049 | 0.8468 | 0.6520 | 0.8248 | 0.7790 | 21.9 |

 For this quantization, we used 1 codebook of 16 bits.
+Results (measured with `lm_eval==4.0`):
 | Model      | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | Winogrande | PiQA | Model size, Gb |
 |------|------|-------|------|------|------|------|------|------|
+|meta-llama/Meta-Llama-3-70B | - | 0.7868 | 0.6596 | 0.8935 | 0.6921	| 0.8444 | 0.8090 | 141.2 |
 |  | 1x16 | 0.7475| 0.6049 | 0.8468 | 0.6520 | 0.8248 | 0.7790 | 21.9 |