Error deploying in colab
So I'm trying to utilize the Colab Env to deploy this model and I've run into an error. I need help fixing it.
NameError Traceback (most recent call last)
in <cell line: 11>()
9 tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
10
---> 11 model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
12 model_basename=model_basename,
13 use_safetensors=True,
7 frames
/usr/local/lib/python3.10/dist-packages/auto_gptq/nn_modules/qlinear/qlinear_cuda_old.py in init(self, bits, group_size, infeatures, outfeatures, bias, use_cuda_fp16, kernel_switch_threshold, trainable)
81 self.kernel_switch_threshold = kernel_switch_threshold
82 self.autogptq_cuda_available = _autogptq_cuda_available
---> 83 self.autogptq_cuda = autogptq_cuda_256
84 if infeatures % 256 != 0 or outfeatures % 256 != 0:
85 self.autogptq_cuda = autogptq_cuda_64
NameError: name 'autogptq_cuda_256' is not defined