How did you exactly quantized this model?

#1
by lesserfield - opened

I tried to quantized the 2.7B model on colab, but no luck and throw me error NotImplementedError: Cannot copy out of meta tensor; no data!

I tried to quantized the 2.7B model on colab, but no luck and throw me error NotImplementedError: Cannot copy out of meta tensor; no data!

2.7B model based on a gpt-neo architecture, it's not GPT-J. You should find quantizer for gpt-neo

lesserfield changed discussion status to closed

Sign up or log in to comment