cicdatopea commited on
Commit
0fc180e
·
verified ·
1 Parent(s): e212ef7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -5,7 +5,7 @@ datasets:
5
 
6
  ## Model Details
7
 
8
- This model is an int4 model with group_size 128 and symmetric quantization of [falcon-three-7b]() generated by [intel/auto-round](https://github.com/intel/auto-round). Load the model with revision `a10e358` to use AutoGPTQ format, with revision `e9aa317` to use AutoAWQ format
9
 
10
  ## How To Use
11
  ### INT4 Inference(CPU/HPU/CUDA)
@@ -13,7 +13,7 @@ This model is an int4 model with group_size 128 and symmetric quantization of [f
13
  from auto_round import AutoRoundConfig ##must import for auto_round format
14
  from transformers import AutoModelForCausalLM, AutoTokenizer
15
 
16
- quantized_model_dir = "OPEA/falcon-three-7b-int4-sym-inc"
17
  tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir)
18
  model = AutoModelForCausalLM.from_pretrained(
19
  quantized_model_dir,
@@ -72,7 +72,7 @@ text = "There is a girl who likes adventure,"
72
  pip3 install lm-eval==0.4.5
73
 
74
  ```bash
75
- auto-round --model "OPEA/falcon-three-7b-int4-sym-inc" --eval --eval_bs 16 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,mmlu
76
  ```
77
 
78
  | Metric | BF16 | INT4 |
@@ -94,7 +94,7 @@ auto-round --model "OPEA/falcon-three-7b-int4-sym-inc" --eval --eval_bs 16 --ta
94
  Here is the sample command to generate the model.
95
  ```bash
96
  auto-round \
97
- --model falcon-three-7b \
98
  --device 0 \
99
  --group_size 128 \
100
  --bits 4 \
 
5
 
6
  ## Model Details
7
 
8
+ This model is an int4 model with group_size 128 and symmetric quantization of [Falcon3-7B-Base](https://huggingface.co/tiiuae/Falcon3-7B-Base) generated by [intel/auto-round](https://github.com/intel/auto-round). Load the model with revision `a10e358` to use AutoGPTQ format, with revision `e9aa317` to use AutoAWQ format
9
 
10
  ## How To Use
11
  ### INT4 Inference(CPU/HPU/CUDA)
 
13
  from auto_round import AutoRoundConfig ##must import for auto_round format
14
  from transformers import AutoModelForCausalLM, AutoTokenizer
15
 
16
+ quantized_model_dir = "OPEA/Falcon3-7B-Base-int4-sym-inc"
17
  tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir)
18
  model = AutoModelForCausalLM.from_pretrained(
19
  quantized_model_dir,
 
72
  pip3 install lm-eval==0.4.5
73
 
74
  ```bash
75
+ auto-round --model "OPEA/Falcon3-7B-Base-int4-sym-inc" --eval --eval_bs 16 --tasks lambada_openai,hellaswag,piqa,winogrande,truthfulqa_mc1,openbookqa,boolq,arc_easy,arc_challenge,mmlu
76
  ```
77
 
78
  | Metric | BF16 | INT4 |
 
94
  Here is the sample command to generate the model.
95
  ```bash
96
  auto-round \
97
+ --model tiiuae/Falcon3-7B-Base \
98
  --device 0 \
99
  --group_size 128 \
100
  --bits 4 \