Intel
/

Qwen2.5-0.5B-Instruct-int4-inc

Model card Files Files and versions Community

n1ck-guo commited on Oct 8, 2024

Commit

f632496

·

verified ·

1 Parent(s): e6cc5a2

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ datasets:
 ## Model Details
-This model is an int4 model with group_size 128 with quantized lm-head of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model
 ## How To Use
@@ -88,7 +88,7 @@ python -m auto_round --model "Intel/Qwen2.5-0.5B-Instruct-int4-inc" --eval --eva
 ```
 | Metric         | BF16   | INT4(group_size 128)   | INT4(group_size 32)   |
-|:--------------:| :----: | :----: |:------:|
 | Avg            | 0.4637 | 0.4496 | 0.4512 |
 | mmlu           | 0.4587 | 0.4440 | 0.4452 |
 | cmmlu          | 0.5036 | 0.4580 | 0.4683 |

 ## Model Details
+This model is an int4 model with group_size 128 of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model
 ## How To Use
 ```
 | Metric         | BF16   | INT4(group_size 128)   | INT4(group_size 32)   |
+|:-------------- | :----: | :----: |:------:|
 | Avg            | 0.4637 | 0.4496 | 0.4512 |
 | mmlu           | 0.4587 | 0.4440 | 0.4452 |
 | cmmlu          | 0.5036 | 0.4580 | 0.4683 |