Приложенный пример не запускается
Ошибка from_pretrained() missing 1 required positional argument: 'quantize_config'
python 3.9.6
transformers 4.3.1
auto_gptq 0.3.1
TypeError Traceback (most recent call last)
Cell In[7], line 4
1 from transformers import AutoTokenizer
2 from auto_gptq import AutoGPTQForCausalLM
----> 4 model = AutoGPTQForCausalLM.from_pretrained('Gaivoronsky/ruGPT-3.5-13B-8bit', device="cuda:0", use_triton=False)
5 tokenizer = AutoTokenizer.from_pretrained('Gaivoronsky/ruGPT-3.5-13B-8bit')
7 request = "Человек: Сколько весит жираф? Помощник: "
TypeError: from_pretrained() missing 1 required positional argument: 'quantize_config'
Try updating auto-gptq
pip install auto-gptq -U
Replacing AutoGPTQForCausalLM.from_pretrained with AutoGPTQForCausalLM.from_quantized helped me. Tried on python3.9 and python 3.11, auto_gptq0.3.1 and auto_gptq0.4.1(from source).