Quantization
#1
by
SinanAkkoyun
- opened
Hi, how did you quantize the 3B model? Which dataset and which code?
Thank you a lot
Hi, how did you quantize the 3B model? Which dataset and which code?
Thank you a lot
Hmm, I use the code from the repo https://github.com/PanQiWei/AutoGPTQ and don't seem to remember very well but it seems I only used 1 sentence for the example to do the quantization.
However, you can refer to the file AutoGPTQ/examples/quantization/basic_usage_wikitext2.py in the repo to use other datasets.
Good luck.