Intel
/

Phi-3-mini-128k-instruct-int4-inc-recipe

Model card Files Files and versions Community

yintongl commited on May 21

Commit

d04b01a

•

1 Parent(s): bb7e659

Update README.md

Files changed (1) hide show

README.md +10 -10

README.md CHANGED Viewed

@@ -55,17 +55,17 @@ lm_eval --model hf --model_args pretrained="Intel/Phi-3-mini-128k-instruct-int4-
 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
-| Avg.           | 0.6365 | 0.6300 |
-| mmlu           | 0.6247 | 0.6237 |
-| lambada_openai | 0.6652 | 0.6433 |
-| hellaswag      | 0.5978 | 0.5859 |
-| winogrande     | 0.7277 | 0.7230 |
-| piqa           | 0.7895 | 0.7846 |
-| truthfulqa_mc1 | 0.3562 | 0.3562 |
 | openbookqa     | 0.3900 | 0.3800 |
-| boolq          | 0.8557 | 0.8489 |
-| arc_easy       | 0.8140 | 0.8199 |
-| arc_challenge  | 0.5444 | 0.5350 |

 | Metric         | BF16   | INT4   |
 | -------------- | ------ | ------ |
+| Avg.           | 0.6364 | 0.6300 |
+| mmlu           | 0.6215 | 0.6237 |
+| lambada_openai | 0.6656 | 0.6433 |
+| hellaswag      | 0.5979 | 0.5859 |
+| winogrande     | 0.7324 | 0.7230 |
+| piqa           | 0.7884 | 0.7846 |
+| truthfulqa_mc1 | 0.3574 | 0.3562 |
 | openbookqa     | 0.3900 | 0.3800 |
+| boolq          | 0.8572 | 0.8489 |
+| arc_easy       | 0.8119 | 0.8199 |
+| arc_challenge  | 0.5418 | 0.5350 |