COVID-19?
I get strange responses when asking to summarize news articles where it invents reasons for particular news or performance saying they are related to the COVID-19 pandemic... this is not a good type of error. I am still testing it as maybe this was a strange one off error. These were the settings: temperature=0.1, repeat_penalty=1.1, top_k=80, top_p=0.5
Overfitting?
Quote from readme
Known Issues and Limitations
Here a list of known issues and bugs. If your issue is not reported here, please open a new issue and describe the problem. See QLORA for any other limitations.
4-bit inference is slow. Currently, our 4-bit inference implementation is not yet integrated with the 4-bit matrix multiplication
Currently, using bnb_4bit_compute_type='fp16' can lead to instabilities.
Make sure that tokenizer.bos_token_id = 1 to avoid generation issues.
Sorry, maybe I am missing something, what information should I derive from the quote from the readme?