a 2b model in gguf format is 10Gb?
That's right! The weights are store in fp32 precision (see here) meaning 2.6B params * 4 bytes per param = ~10GB.
2.6B params * 4 bytes per param = ~10GB
Hi @JohnLaw , Could you please confirm if this issue is resolved for above Answered . We can close this issue or else if you have any concerns let us know. Thank you.
· Sign up or log in to comment
