would be great to have the example code for the attention mask and the pad token id
hi there, big fan of stable lm since the alpha :-)
this one seems great, but i'm getting warnings about the attention mask and the pad token id :-)
will dig in as always but it would be a quick fix on the example code.
congrats on the cool release ! 🚀
hi there, big fan of stable lm since the alpha :-)
this one seems great, but i'm getting warnings about the attention mask and the pad token id :-)
will dig in as always but it would be a quick fix on the example code.
congrats on the cool release ! 🚀
Adding pad_token_id=tokenizer.pad_token_id, fixes it:
tokens = model.generate(
inputs.to(model.device),
pad_token_id=tokenizer.pad_token_id,
max_new_tokens=1024,
temperature=0.8,
do_sample=True
(Tried with transformers version 4.35.2)