Limit of tokens

#4
by francostan - opened

Hi everyone, im facing this problems that all the ai response generated through generate() are limited on 256 tokens:

Prompt: 707 tokens, 180.446 tokens-per-sec
Generation: 256 tokens, 20.440 tokens-per-sec
Peak memory: 4.440 GB
Respuesta generada:
...

Anyone know how to change this limit, should be on load() ?

Sign up or log in to comment