qwp4w3hyb
/

gemma-2-9b-it-iMat-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

qwp4w3hyb commited on Jun 28

Commit

1862c34

•

1 Parent(s): 9f7f2ec

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -14,7 +14,7 @@ base_model: google/gemma-2-9b-it
 # Quant Infos
 - quants done with an importance matrix for improved quantization loss
-- Currently requantizing ggufs & imatrix from bf16 for "optimal" accuracy loss
   - initial version was based on f32 gguf provided by google
   - WIP new version should have better metadata as its quantized from hf safetensors with llama.cpp
 - Wide coverage of different gguf quant types from Q\_8\_0 down to IQ1\_S

 # Quant Infos
 - quants done with an importance matrix for improved quantization loss
+- Currently requantizing ggufs & imatrix from bf16
   - initial version was based on f32 gguf provided by google
   - WIP new version should have better metadata as its quantized from hf safetensors with llama.cpp
 - Wide coverage of different gguf quant types from Q\_8\_0 down to IQ1\_S