csdc-atl
/

Baichuan2-13B-Chat-GPTQ-Int4

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

xiaotinghe commited on Sep 19, 2023

Commit

d9e7596

•

1 Parent(s): c95623e

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -62,7 +62,7 @@ pip3 install .
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 from transformers.generation.utils import GenerationConfig
-model_name_or_path = "csdc-atl/Baichuan2-7B-Chat-Int4"
 model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
                                              torch_dtype=torch.float16,
                                              device_map="auto",

 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 from transformers.generation.utils import GenerationConfig
+model_name_or_path = "csdc-atl/Baichuan2-13B-Chat-Int4"
 model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
                                              torch_dtype=torch.float16,
                                              device_map="auto",