qwp4w3hyb
/

gemma-2-9b-it-iMat-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

qwp4w3hyb commited on Jun 28

Commit

40264e7

•

1 Parent(s): 214a857

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -15,8 +15,9 @@ base_model: google/gemma-2-9b-it
 - quants done with an importance matrix for improved quantization loss
 - Currently requantizing ggufs & imatrix from bf16
-  - initial version was based on f32 gguf provided by google
   - WIP new version should have better metadata as its quantized from hf safetensors with llama.cpp
 - Wide coverage of different gguf quant types from Q\_8\_0 down to IQ1\_S
   - experimental custom quant types
     - `_L` with `--output-tensor-type f16 --token-embedding-type f16` (same as bartowski's)

 - quants done with an importance matrix for improved quantization loss
 - Currently requantizing ggufs & imatrix from bf16
+  - initial version was based on f32 gguf provided by google, which has various issues
   - WIP new version should have better metadata as its quantized from hf safetensors with llama.cpp
+    - still updating atm, _L & _XL are already the new version, other quants will update over the day
 - Wide coverage of different gguf quant types from Q\_8\_0 down to IQ1\_S
   - experimental custom quant types
     - `_L` with `--output-tensor-type f16 --token-embedding-type f16` (same as bartowski's)