GGUFs eventually ?
#1
by
HMasaki
- opened
I like to use ollama and it's easiest to set them up when they're GGUFs ideally Q8 is there a plan to eventually do some GGUFs of this one?
I tried converting it, but something seems to be wrong. I'm trying to solve the problem.
The BPE pre-tokenizer was not recognized!
There are 2 possible reasons for this:
- the model has not been added to convert_hf_to_gguf_update.py yet
- the pre-tokenization config has changed upstream
Check your model files and convert_hf_to_gguf_update.py and update them accordingly.
Use with ollama
You can use huihui_ai/falcon3-abliterated directly, It's possible for the output to be empty.
ollama run huihui_ai/falcon3-abliterated:10b
That works thank you
Using gguf by bartowski. Indeed empty output (not always). Looks like the problem is q4_0.gguf
It sometimes appears, and it goes away with a try.