Update README.md
Browse files
README.md
CHANGED
@@ -18,6 +18,33 @@ Inference of this model is compatible with AutoGPTQ's Kernel.
|
|
18 |
|
19 |
|
20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
### Evaluate the model
|
22 |
|
23 |
Install [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness.git) from source, we used the git id 96d185fa6232a5ab685ba7c43e45d1dbb3bb906d
|
@@ -44,31 +71,6 @@ lm_eval --model hf --model_args pretrained="Intel/Phi-3-mini-128k-instruct-int4-
|
|
44 |
|
45 |
|
46 |
|
47 |
-
### Reproduce the model
|
48 |
-
|
49 |
-
Here is the sample command to reproduce the model
|
50 |
-
|
51 |
-
```bash
|
52 |
-
git clone https://github.com/intel/auto-round
|
53 |
-
cd auto-round/examples/language-modeling
|
54 |
-
pip install -r requirements.txt
|
55 |
-
python3 main.py \
|
56 |
-
--model_name microsoft/Phi-3-mini-128k-instruct \
|
57 |
-
--device 0 \
|
58 |
-
--group_size 128 \
|
59 |
-
--bits 4 \
|
60 |
-
--iters 200 \
|
61 |
-
--seqlen 4096 \
|
62 |
-
--minmax_lr 0.01 \
|
63 |
-
--deployment_device 'gpu' \
|
64 |
-
--gradient_accumulate_steps 2 \
|
65 |
-
--train_bs 4 \
|
66 |
-
--output_dir "./tmp_autoround" \
|
67 |
-
|
68 |
-
```
|
69 |
-
|
70 |
-
|
71 |
-
|
72 |
|
73 |
## Caveats and Recommendations
|
74 |
|
|
|
18 |
|
19 |
|
20 |
|
21 |
+
### Reproduce the model
|
22 |
+
|
23 |
+
Here is the sample command to reproduce the model
|
24 |
+
|
25 |
+
```bash
|
26 |
+
git clone https://github.com/intel/auto-round
|
27 |
+
cd auto-round/examples/language-modeling
|
28 |
+
pip install -r requirements.txt
|
29 |
+
python3 main.py \
|
30 |
+
--model_name microsoft/Phi-3-mini-128k-instruct \
|
31 |
+
--device 0 \
|
32 |
+
--group_size 128 \
|
33 |
+
--bits 4 \
|
34 |
+
--iters 200 \
|
35 |
+
--seqlen 4096 \
|
36 |
+
--minmax_lr 0.01 \
|
37 |
+
--deployment_device 'gpu' \
|
38 |
+
--gradient_accumulate_steps 2 \
|
39 |
+
--train_bs 4 \
|
40 |
+
--output_dir "./tmp_autoround" \
|
41 |
+
|
42 |
+
```
|
43 |
+
|
44 |
+
|
45 |
+
|
46 |
+
|
47 |
+
|
48 |
### Evaluate the model
|
49 |
|
50 |
Install [lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness.git) from source, we used the git id 96d185fa6232a5ab685ba7c43e45d1dbb3bb906d
|
|
|
71 |
|
72 |
|
73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
74 |
|
75 |
## Caveats and Recommendations
|
76 |
|