Can you produce a quantized 2.4bpw model of this model?
#1
by
xldistance
- opened
@async0x42 24GB of video memory can only run 2.4bpw quantization
xldistance
changed discussion title from
Can you produce a quantized 2.25bpw model of this model?
to Can you produce a quantized 2.4bpw model of this model?
Sure thing, was on vacation before so I wasn't able to, but I'm starting the process now; will keep you posted
@xldistance uploading 2.4bpw here: https://huggingface.co/async0x42/Rombos-LLM-V2.5-Qwen-72b-exl2_2.4bpw
async0x42
changed discussion status to
closed
@xldistance uploading 2.4bpw here: https://huggingface.co/async0x42/Rombos-LLM-V2.5-Qwen-72b-exl2_2.4bpw
Thank you so much.