Still getting garbage in output with mlx 0.11.1 and latest mlx-examples.

#1
by ivanfioravanti - opened
MLX Community org

Still not working downloading this quantized model. Anyone lese facing same issue?

❯ python -m mlx_lm.generate --model mlx-community/Meta-Llama-3-70B-Instruct-4bit --prompt "Write a snake game in python" --temp 0
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

Prompt: <|begin_of_text|><|start_header_id|>user<|end_header_id|>

Write a snake game in python<|eot_id|><|start_header_id|>assistant<|end_header_id|>

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

MLX Community org

Never mind! Second download worked!

ivanfioravanti changed discussion status to closed

Sign up or log in to comment