Spaces:
Running
on
A10G
Running
on
A10G
Doesn't work for Phi-3 models
#47
by
smcleod
- opened
For example, trying to create a Q6_K from microsoft/Phi-3-mini-128k-instruct results in:
Error: Error converting to fp16: b'Traceback (most recent call last):\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 2807, in \n main()\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 2787, in main\n model_class = Model.from_model_architecture(hparams["architectures"][0])\n File "/home/user/app/llama.cpp/convert-hf-to-gguf.py", line 216, in from_model_architecture\n raise NotImplementedError(f'Architecture {arch!r} not supported!') from None\nNotImplementedError: Architecture 'Phi3ForCausalLM' not supported!\n'
That's odd - let me look into it.
This actually works fine for me: https://huggingface.co/reach-vb/Phi-3-mini-4k-instruct-Q8_0-GGUF
I think the reason for your error was that the official support had not been merged into llama.cpp at that time: https://github.com/ggerganov/llama.cpp/pull/6852
(closing this for now)
reach-vb
changed discussion status to
closed