Upload gguf-imat-llama-3.py
#30
by
SolidSnacke
- opened
I advise you to create a separate branch for the version of this file, since the most likely thing in the future is to solve the problem with running bf16.gguf on the gpu, due to which it is impossible to create imatrix.dat. Therefore, we had to create imatrix.dat from f16.gguf and quantized models from bf16.gguf.
Thanks! Will keep as an additional file.
FantasiaFoundry
changed pull request status to
merged