Exl2 quants?
#1
by
ElvisM
- opened
I can never get GGUFs to work on WebUI. It always gives me an error. Plus, this model is small enough to be run entirely on a GPU. I still have no idea why GGUF is such a popular format.
Main reason is it's just so much slower, but I'll make this one since you asked 🤗
oops, forgot to make it public.. here you go: https://huggingface.co/bartowski/LongWriter-llama3.1-8b-exl2