Could not find the quantized model in .pt or .safetensors format, exiting...
#14
by
donflopez
- opened
When I try to run oobabooga with this model I get the error on the following error:
Could not find the quantized model in .pt or .safetensors format, exiting...
Command used:
python server.py --model elinas_alpaca-30b-lora-int4 --wbits 4 --groupsize 128 --chat --model_type llama --listen
Please read this, you need to switch to another fork due to breaking changes, and this only works on cuda https://huggingface.co/elinas/alpaca-30b-lora-int4#important---update-2023-04-05
elinas
changed discussion status to
closed