FinReport

Runtime error

No1r97 commited on Apr 19

Commit

d56335e

•

1 Parent(s): a9aaf24

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -23,7 +23,7 @@ base_model = AutoModelForCausalLM.from_pretrained(
     token=access_token,
     trust_remote_code=True,
     device_map="auto",
-    torch_dtype=torch.float16,
     offload_folder="offload/"
 )
 model = PeftModel.from_pretrained(

     token=access_token,
     trust_remote_code=True,
     device_map="auto",
+    load_in_8bit=True,
     offload_folder="offload/"
 )
 model = PeftModel.from_pretrained(