rmdhirr
/

Bepis_9B-8.0bpw-h8-exl2

Text Generation

Inference Endpoints

text-generation-inference

8-bit precision

Model card Files Files and versions Community

Ramadhirra commited on Apr 7

Commit

46b5c87

•

1 Parent(s): 039e973

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -28,7 +28,7 @@
     "quantization_config": {
         "quant_method": "exl2",
         "version": "0.0.17",
-        "bits": 8.0,
         "head_bits": 8,
         "calibration": {
             "rows": 100,

     "quantization_config": {
         "quant_method": "exl2",
         "version": "0.0.17",
+        "bits": 8,
         "head_bits": 8,
         "calibration": {
             "rows": 100,