neuralmagic
/

Phi-3.5-mini-instruct-FP8-KV

Text Generation

compressed-tensors

Model card Files Files and versions Community

mgoin commited on Sep 27, 2024

Commit

aed827e

·

verified ·

1 Parent(s): 3a3f45d

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -10,7 +10,7 @@
     "AutoModelForCausalLM": "modeling_phi3.Phi3ForCausalLM"
   },
   "bos_token_id": 1,
-  "compression_config": {
     "config_groups": {
       "group_0": {
         "input_activations": {

     "AutoModelForCausalLM": "modeling_phi3.Phi3ForCausalLM"
   },
   "bos_token_id": 1,
+  "quantization_config": {
     "config_groups": {
       "group_0": {
         "input_activations": {