Hugging Face's logo

ISTA-DASLab
/
Meta-Llama-3.1-8B-Instruct-AQLM-PV-2Bit-1x16-hf

Text Generation
Transformers
Safetensors
llama
aqlm
facebook
meta
llama-3
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
2
SpiridonSunRotator commited on
Commit
59a8e38
·
verified ·
1 Parent(s): d52aef2

Updated metrics for new revision of model

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -17,8 +17,12 @@ Results:
17
  | Model | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | PiQA | Winogrande | Model size, Gb |
18
  |------|------|-------|------|------|------|------|------|------|
19
  |meta-llama/Meta-Llama-3.1-8B-Instruct| None | 0.6817 | 0.5162 | 0.8186 | 0.5909 | 0.8014 | 0.7364 | 16.1 |
20
- | | 1x16g8 | 0.5917 | 0.4377 | 0.7681 | 0.5233 | 0.7748 | 0.6867 | 4.1 |
21
 
22
  **Note**
23
 
24
- We used `lm-eval=0.4.0` for evaluation.
 
 
 
 
 
17
  | Model | Quantization | MMLU (5-shot) | ArcC| ArcE| Hellaswag | PiQA | Winogrande | Model size, Gb |
18
  |------|------|-------|------|------|------|------|------|------|
19
  |meta-llama/Meta-Llama-3.1-8B-Instruct| None | 0.6817 | 0.5162 | 0.8186 | 0.5909 | 0.8014 | 0.7364 | 16.1 |
20
+ | | 1x16g8 | 0.5966 | 0.4667 | 0.7790 | 0.5582 | 0.7878 | 0.7001 | 4.1 |
21
 
22
  **Note**
23
 
24
+ We used `lm-eval=0.4.0` for evaluation.
25
+
26
+ **UPD (09.08.2024)**
27
+
28
+ Uploaded new version finetuned on more data for longer with better quality.