Quantization?

#7
by PacmanIncarnate - opened

Any chance you could quantize the model? Could open it up to a lot more users.

Doctor-Shotgun/YuE-s1-7B-anneal-en-cot-exl2
Doctor-Shotgun/YuE-s2-1B-general-exl2

https://github.com/sgsdxzy/YuE-exllamav2

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment