MLX / MPS users out of luck and can't use this model with VLLM
#4
by
kronosprime
- opened
VLLM doesn't support Apple machines, first requested back in October 2023: https://github.com/vllm-project/vllm/issues/1441
They closed the issue. NVIDIA is a stingy greed freak about RAM, and with Apple we get unified memory, so we can build servers with 512GB ram usable with our GPUs.
Is there another way to load Pixtral without VLLM?
Thanks
Any updates? Wanna run this on my Mac!
It looks like someone is working on a solution:
https://github.com/Blaizzy/mlx-vlm
Pixtral is one of the supported models.
I wish I had a Mac to experiment with.