iMatrix GGUFs for https://huggingface.co/01-ai/Yi-9B-200K

iMatrix generated with Kalomaze's semi-random groups_merged.txt

Included fp16 GGUF as well, just to save a conversion step if anybody else feels like doing additional quants.

Downloads last month
16
GGUF
Model size
8.83B params
Architecture
llama
Inference Examples
Unable to determine this model's library. Check the docs .