Undi95
/

Meta-Llama-3-70B-Instruct-hf

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Undi95 commited on May 10

Commit

8db8ac3

•

1 Parent(s): 62d6935

Upload README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -304,7 +304,7 @@ pipeline = transformers.pipeline(
     "text-generation",
     model=model_id,
     model_kwargs={"torch_dtype": torch.bfloat16},
-    device="auto",
 )
 messages = [

     "text-generation",
     model=model_id,
     model_kwargs={"torch_dtype": torch.bfloat16},
+    device_map="auto",
 )
 messages = [