mistral
finetuned
quantized
AWQ
instruct
conversational
text-generation-inference
finetune
chatml
DPO
RLHF
gpt4
synthetic data
distillation
awq
Shaun Prince
adding quant config
2d76893
|
{ |
|
"zero_point": true, |
|
"q_group_size": 128, |
|
"w_bit": 4, |
|
"version": "GEMM" |
|
} |