Why does llama-2-7b-chat.ggmlv3.q2_K.bin always load using Metal?
#19
by
Auxon
- opened
I was testing llama-2-7b-chat.ggmlv3.q2_K.bin using Metal, but then decided to test with CPU but it always loads using Metal, while other models use the CPU ... strange. I made sure to remove and reinstall all required modules. Wondering if anyone knows why? I'm using GPT4All (API) in this case, but was using llama-cpp-python when testing using Metal.