Hugging Face's logo

IlyaGusev
/
saiga_13b_lora

Text Generation
Russian
conversational
Model card Files Community
3
IlyaGusev commited on
Commit
07c783b
•
1 Parent(s): 38201e0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -84,6 +84,7 @@ config = PeftConfig.from_pretrained(MODEL_NAME)
84
  model = AutoModelForCausalLM.from_pretrained(
85
  config.base_model_name_or_path,
86
  load_in_8bit=True,
 
87
  device_map="auto"
88
  )
89
  model = PeftModel.from_pretrained(model, MODEL_NAME)
 
84
  model = AutoModelForCausalLM.from_pretrained(
85
  config.base_model_name_or_path,
86
  load_in_8bit=True,
87
+ torch_dtype=torch.float16,
88
  device_map="auto"
89
  )
90
  model = PeftModel.from_pretrained(model, MODEL_NAME)