Potential bug when batch inferencing with left padding.
#26
by
Snorlax
- opened
In line 213 of modeling_minicpmv.py, after performing left padding on input_ids, the image_bound is still using the result from before the left padding. This seems to cause image features to be put to the wrong position in line 153).
We haven't tested batch inference yet. This is indeed wrong during batch inference. You need to fix the image_bound here and pass in a correct attention_mask.
Or you can reset the image_bound by special token