The versions should be marked by their GPU support
#1
by
seedmanc
- opened
Which ones work with GPU inference? I haven't tested all of them but I have a feeling that only the basic Q4 does. It would be a waste having to DL all of them for the users seeking hardware-accelerated inference.
...they all do.