Attention mask and pad token id warning
#28
by
nottakumasato
- opened
Hi,
I have been getting the below warning message when running inference. How can I fix it editing the example code in the docs?
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Hello
I don't know if you still need help, but if you do you can suppress the warnings by setting pad_token_id to eos_token_id in the generate method from the example code.
For example, you can update the example code with
generated_ids = model.generate(model_inputs, pad_token_id=tokenizer.eos_token_id, max_new_tokens=1000, do_sample=True).
Hope this helps:)