Hugging Face's logo

phatjk
/
vinallama-7b-AWQ

Text Generation
Transformers
Safetensors
llama
text-generation-inference
Inference Endpoints
4-bit precision
awq
Model card Files Community
phatjk commited on
Commit
d8bd191
•
1 Parent(s): f7e6035

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md ADDED
@@ -0,0 +1 @@
 
 
1
+ quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM" }