mayank-mishra's picture
add mmodel
a9bbb8c
metadata
license: openrail

GPTQ-for-SantaCoder

Visit GPTQ-for-SantaCoder for instructions on how to use the model weights here. If you want 4-bit weights, visit santacoder-GPTQ-4bit-128g.

Results

SantaCoder Bits group-size memory(MiB) wikitext2 ptb c4 stack checkpoint size(MB)
FP32 32 - 4344.722 24.927 38.574 27.779 2.619 4394
BF16 16 - 2173.680 24.960 38.597 27.794 2.621 2195
GPTQ 8 -1 1396.548 24.936 38.592 27.785 2.619 1411
GPTQ 4 -1 911.384 26.581 40.717 29.232 2.658 913
GPTQ 3 -1 - 11761.473 7273.338 9124.941 2485.844 789
GPTQ 2 -1 - 67976.797 68994.484 73294.438 45370.488 649

License

The model is licenses under the CodeML Open RAIL-M v0.1 license. You can find the full license here.

Acknowledgements

Thanks to everyone in BigCode who worked so hard to create these code models.