3b, 8b, and 34b versions of GGUF?
#1
by
tombenninger
- opened
Are currently I see the tensor data version of Granite for 3b, 8b, 20b, and 34b, but only a 20b version packaged in GGUF. Will the other size models be packaged in GGUF?
Ill be releasing 34b today abd imthe instruct of 20b
Some folks are working on making the 3 and 8b work, more details here: https://github.com/ggerganov/llama.cpp/issues/7116#issuecomment-2100061526
mayank-mishra
changed discussion status to
closed