The versions should be marked by their GPU support

#1
by seedmanc - opened

Which ones work with GPU inference? I haven't tested all of them but I have a feeling that only the basic Q4 does. It would be a waste having to DL all of them for the users seeking hardware-accelerated inference.

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment