CUDA Error when generate
#9
by
heungson
- opened
This comment has been hidden
heungson
changed discussion status to
closed
heungson
changed discussion title from
CUDA error: CUBLAS_STATUS_NOT_INITIALIZED
to CUDA Error when using StoppingCriteria
heungson
changed discussion title from
CUDA Error when using StoppingCriteria
to CUDA Error when generate
When generate() is called, the model raises the following error:
CUDA error - CUBLAS_STATUS_NOT_INITIALIZED.
It seems that the error occurs due to an issue when loading the model and can sometimes be resolved by either resizing the token embeddings with
model.resize_token_embeddings(len(tokenizer))
or by reloading the model.
The model is running on GPU GV100 (32GB RAM) and I didn't add any new token to the tokenizer.
heungson
changed discussion status to
open