fixed num_tokens to use length of tokenization, increased max token limit 6ede84a Mazin Karjikar commited on Sep 17, 2024
added token limits and error handling for long prompts 2c0daa4 Mazin Karjikar commited on Sep 17, 2024