Still getting garbage in output with mlx 0.11.1 and latest mlx-examples.
#1
by
ivanfioravanti
- opened
Still not working downloading this quantized model. Anyone lese facing same issue?
❯ python -m mlx_lm.generate --model mlx-community/Meta-Llama-3-70B-Instruct-4bit --prompt "Write a snake game in python" --temp 0
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Prompt: <|begin_of_text|><|start_header_id|>user<|end_header_id|>
Write a snake game in python<|eot_id|><|start_header_id|>assistant<|end_header_id|>
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
Never mind! Second download worked!
ivanfioravanti
changed discussion status to
closed