Spaces:
Running
on
A100
System prompt and Generation Config for better results?
I have tried a bunch a prompts but it didn't yeid good results in Nepali , it took too much time using transformers.
ANy suggestion for prompting to make conversational chatbot in Nepali language ?
Ps: the zephyr chat really performs good in Nepali . Bud didn't work locally using my own promts
like "You are an expert conversation chatbot that can reply in Nepali language fluently, Try to answer the asked question."
Hey @Saugatkafley we didn't use any multilingual data when training so Zephyr 7B Beta's ability to converse in Nepali is largely constrained by whatever Mistral used in pretraining the base model. When you say Zephyr Chat works well for Nepali but not locally, perhaps you can try using the same parameters we used for sampling in the demo:
{ "temperature": 0.7, "top_p": 0.95, "repetition_penalty": 1.2, "top_k": 50, "max_new_tokens": 1024, "stop": ["</s>", "<|>"]}
Btw we have a Colab example for the alpha model that should also work for the beta one: https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha/blob/main/colab-demo.ipynb