Could we possibly have a Q4_K_M? It allows for 16k context in 8gb of vram :3
aight was busy the whole day, will quant
https://huggingface.co/Sao10K/Solus-m7-GGUF/resolve/main/Solus-m7.q4_K_M.gguf
· Sign up or log in to comment