Update README.md
Browse files
README.md
CHANGED
@@ -1,8 +1,8 @@
|
|
1 |
## Model Card Details
|
2 |
|
3 |
-
This is
|
4 |
|
5 |
-
|
6 |
|
7 |
Here is the sample command to reproduce the model
|
8 |
|
@@ -25,7 +25,7 @@ python3 main.py \
|
|
25 |
|
26 |
|
27 |
|
28 |
-
|
29 |
|
30 |
1 git clone https://github.com/intel/auto-round
|
31 |
|
@@ -69,7 +69,7 @@ text = "Tell me something about Intel. "
|
|
69 |
|
70 |
|
71 |
|
72 |
-
|
73 |
|
74 |
lm-eval 0.4.2 eval_bs 16
|
75 |
|
|
|
1 |
## Model Card Details
|
2 |
|
3 |
+
This is a recipe of int4 model with group_size 128 for meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round).
|
4 |
|
5 |
+
## Reproduce the model
|
6 |
|
7 |
Here is the sample command to reproduce the model
|
8 |
|
|
|
25 |
|
26 |
|
27 |
|
28 |
+
## Inference on CPU/HPU//CUDA
|
29 |
|
30 |
1 git clone https://github.com/intel/auto-round
|
31 |
|
|
|
69 |
|
70 |
|
71 |
|
72 |
+
## Evaluate the model
|
73 |
|
74 |
lm-eval 0.4.2 eval_bs 16
|
75 |
|