Run with 24gb vram?
#1
by
Doomed1986
- opened
I notice this is 26.16gb roughly. I take it this wont run on a 4090 without offloading to cpu (which i havent done before/want to avoid)? Is there a quantised version or anything suitable for 24gb vram?
I notice this is 26.16gb roughly. I take it this wont run on a 4090 without offloading to cpu (which i havent done before/want to avoid)? Is there a quantised version or anything suitable for 24gb vram?
Maybe you could try this one, a quantized version made by someone else:
https://huggingface.co/s3nh/diffnamehard-Psyfighter2-Noromaid-ties-13B-GGUF/tree/main