NOTE: DO NOT USE THESE QUANTS, suffers from corruption issues!
(this repo is only public atm for debugging purposes)
Quick and dirty GGUF quants. Maybe some iMatrix soon. BF16 conversion included in this repo.
split:
split -b 40G -d --additional-suffix=.part shisa-v1-llama3-70b.bf16.gguf shisa-v1-llama3-70b.bf16.gguf
put it back together:
cat shisa-v1-llama3-70b.bf16.gguf*.part > shisa-v1-llama3-70b.bf16.gguf
insure order
cat $(ls -v shisa-v1-llama3-70b.bf16.gguf*.part) > shisa-v1-llama3-70b.bf16.gguf