Unable to get complete output and seems pending while running this project

#11
by zangyun - opened

I cloned and deployed this project locally. But when I run it I find there are two issues:

1.model always doesn't provide me complete answer.I try to track the ouput of model and find that it seems to repeat to print empty string until length of output exceeds max_new_token like this:

no complete answer.png

2.model output duplicate content until length exceeds max_new_token.
instruction:write a c++ code to do quick sort.

duplicate.png

current configuration:Top-k:10 Top-p:0.1 Temperature:0.7 Max new tokens:1024 I tried to fine-tune these parameters with little success.
Could anybody share any ideas to address this strange issue?

Sign up or log in to comment