anon8231489123

/

gpt4-x-alpaca-13b-native-4bit-128g

like 733

When i try to send a message to the ai i get this error listed down below and in the website the ai just replies with a blank black box

i'm using my computer cpu and here are the params i use CMD_FLAGS = '--chat --cpu --wbits 4 --groupsize 128 --pre_layer 28'

and here is the error:

Traceback (most recent call last):
  File "C:\AI\oobabooga_windows\text-generation-webui\modules\callbacks.py", line 55, in gentask
    ret = self.mfunc(callback=_callback, *args, **self.kwargs)
  File "C:\AI\oobabooga_windows\text-generation-webui\modules\text_generation.py", line 289, in generate_with_callback
    shared.model.generate(**kwargs)
  File "C:\AI\oobabooga_windows\installer_files\env\lib\site-packages\auto_gptq\modeling\_base.py", line 422, in generate
    with torch.inference_mode(), torch.amp.autocast(device_type=self.device.type):
  File "C:\AI\oobabooga_windows\installer_files\env\lib\site-packages\auto_gptq\modeling\_base.py", line 411, in device
    device = [d for d in self.hf_device_map.values() if d not in {'cpu', 'disk'}][0]
IndexError: list index out of range
Output generated in 0.44 seconds (0.00 tokens/s, 0 tokens, context 37, seed 486230895)

sorry if i'm missing something obvious but i'm new to this