Method to get 8bit quantized model
#1
by
kitaharatomoyo
- opened
Can you tell me how you get the 8bit quantized model from falcon-7b?
I want to get my own 8bit quantized model from finetuned falcon-7b model.
- transformers from_pretrained: load_in_8bit=True
- push to hub