I have tried this model's inference with bfloat16 type and have not faced any problems. Is it fine to use bfloat16 dtype for this model's inference, without much effect on score?
yeah, it should be ok @lkm1
we have run scores in bf16
ok, thanks @mayank-mishra !
· Sign up or log in to comment
