even more formatting

README.md

@@ -11,15 +11,15 @@ Try it out at [Reka Space](https://space.reka.ai).
 
 ## Quickstart
 
-For easiet consumption, the model is released in a Llama-compatible format. Feel free to use any library compatible with Llama to run the model.
+For easing deployment, the model is released in a Llama-compatible format. You may use any library compatible with Llama to run the model.
 
 ### Via Hugging Face
 
 ```python
 import transformers
 
-tokenizer = transformers.AutoTokenizer.from_pretrained("reka/reka-flash-3")
-model = transformers.AutoModelForCausalLM.from_pretrained("reka/reka-flash-3", torch_dtype='auto', device_map='auto')
+tokenizer = transformers.AutoTokenizer.from_pretrained("RekaAI/reka-flash-3")
+model = transformers.AutoModelForCausalLM.from_pretrained("RekaAI/reka-flash-3", torch_dtype='auto', device_map='auto')
 
 prompt = {"role": "user", "content": "Write a poem about large language model."}
 text = tokenizer.apply_chat_template([prompt], tokenize=False, add_generation_prompt=True)
@@ -38,12 +38,14 @@ docker run --rm -it --network=host --gpus '"device=0"' -v  --shm-size=10.24gb vl
 
 ### Prompt Format
 
-Reka Flash 3 uses cl100k_base tokenizer and adds no additional special tokens. The general prompt format is as follows:
+Reka Flash 3 uses cl100k_base tokenizer and adds no additional special tokens. Its prompt format is as follows:
 
 ```
 human: this is round 1 prompt <sep> assistant: this is round 1 response <sep> ...
 ```
 
+Generation should stop on seeing the string `<sep>` or seeing the special token `<|endoftext|>`.
+
 System prompt can be added by prepending to the first user round.
 
 ```
@@ -52,6 +54,8 @@ human: You are a friendly assistant blah ... this is round 1 user prompt <sep> a
 
 And for multi-round conversations, it is recommended to drop the Chain-Of-Thought reasoning traces in the previous assistant round to save tokens for the model to think.
 
+If you are using HF or vLLM, the built-in chat_template shall handle prompt formatting automatically.
+
 ### Budget Forcing
 
 Reka Flash thinks before it produces an output. We use <reasoning> </reasoning> tags to indicate the beginning and the end of its thinking process. For some problems, the model might think for a long time. You can make the model to stop its thinking process by forcing it to output </reasoning> after a certain number of steps. We observe such a budget forcing mechanism will still produce a reasonable output. We show performance on AIME-2024 (cons@16) for various budgets below.

generation_config.json

@@ -0,0 +1,10 @@
+{
+    "bos_token_id": 100257,
+    "do_sample": true,
+    "eos_token_id": 100257,
+    "pad_token_id": 100257,
+    "temperature": 0.6,
+    "top_k": 1024,
+    "top_p": 0.95
+}
+