SaisExperiments/Gemma-2-2B-Opus-Instruct
#270
by
SaisExperiments
- opened
This is my first "proper" model i've trained and would like to test it with gguf / (probably more likely) if there is any errors in quantization i need to fix.
I'd quant it myself but my hardware is doing more training :3
And i don't know how to make the arm quants either 😸
SaisExperiments/Gemma-2-2B-Opus-Instruct
We quant proper and improper models alike :) I will report how it goes.
The arm quants are trivial to make, namely the same as the others. Instead of specifying Q4_K_S as type you specify Q4_0_4_8 or so. All you need is a current version of llama.cpp.
Good luck to us all :)
mradermacher
changed discussion status to
closed
It converted and the imatrix is being generated. Chances that something goes wrong at this stage are slim. TL;DR: all seems fine :)
Thank you!