phatjk
/

vinallama-7b-AWQ

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

vinallama-7b-AWQ / README.md

phatjk's picture

Create README.md

d8bd191 about 1 year ago

|

history blame contribute delete

89 Bytes

quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM" }