Error with dtype=torch.float16.
#10
by
Cheshire94
- opened
In Qwen-7B-Chat/qwen_generation_utils.py
line 349： scores[i, self.eos_token_id] = float(2**30)
When using torch_dtype=torch.float16, this line above trigger the following error:
RuntimeError: value cannot be converted to type at::Half without overflow
Maybe consider changing it to torch.finfo(torch.float16).max when the dtype is set to torch.float16 and keeping it to float(2**30) for torch.float32?
We manually set it to float(2**15), try again?
Thank you for raising this problem!
It has been fixed in main. I'll just close this for now. If the problem persists, please open a new issue.
jklj077
changed discussion status to
closed