How to use this model with python code `ctransformers`

by anhlht - opened Jun 9

Jun 9

llm = AutoModelForCausalLM.from_pretrained(
"QuantFactory/Meta-Llama-3-8B-Instruct-GGUF", model_file="Meta-Llama-3-8B-Instruct.Q8_0.gguf", model_type="llama", gpu_layers=50)
I tried this code but got an error

meyers1

Aug 16

u should use llamaCpp

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment