How did you exactly quantized this model?
#1
by
lesserfield
- opened
I tried to quantized the 2.7B model on colab, but no luck and throw me error NotImplementedError: Cannot copy out of meta tensor; no data!
I tried to quantized the 2.7B model on colab, but no luck and throw me error
NotImplementedError: Cannot copy out of meta tensor; no data!
2.7B model based on a gpt-neo architecture, it's not GPT-J. You should find quantizer for gpt-neo
Thank you
lesserfield
changed discussion status to
closed