Text Generation
PyTorch
causal-lm
rwkv
BlinkDL commited on
Commit
1e12a22
1 Parent(s): 5f88540

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -6
README.md CHANGED
@@ -38,8 +38,7 @@ The differences between World & Raven:
38
  * set pipeline = PIPELINE(model, "rwkv_vocab_v20230424") instead of 20B_tokenizer.json (EXACTLY AS WRITTEN HERE. "rwkv_vocab_v20230424" is included in rwkv 0.7.4+)
39
  * use Question/Answer or User/AI or Human/Bot for chat. **DO NOT USE Bob/Alice or Q/A**
40
 
41
- For small models, use **fp32** for first layer (will overflow in fp16 at this moment - fixable in future), or bf16 if you have 30xx/40xx GPUs.
42
- Example strategy: cuda fp32 *1 -> cuda fp16
43
 
44
  NOTE: the new greedy tokenizer (https://github.com/BlinkDL/ChatRWKV/blob/main/tokenizer/rwkv_tokenizer.py) will tokenize '\n\n' as one single token instead of ['\n','\n']
45
 
@@ -60,11 +59,11 @@ Response:
60
 
61
  A good chat prompt (replace \n\n in xxx to \n):
62
  ```
63
- Question: hi
64
 
65
- Answer: Hi. I am your assistant and I will provide expert full response in full details. Please feel free to ask any question and I will always answer it.
66
 
67
- Question: xxx
68
 
69
- Answer:
70
  ```
 
38
  * set pipeline = PIPELINE(model, "rwkv_vocab_v20230424") instead of 20B_tokenizer.json (EXACTLY AS WRITTEN HERE. "rwkv_vocab_v20230424" is included in rwkv 0.7.4+)
39
  * use Question/Answer or User/AI or Human/Bot for chat. **DO NOT USE Bob/Alice or Q/A**
40
 
41
+ For 0.1/0.4/1.5B models, use **fp32** for first layer (will overflow in fp16 at this moment - fixable in future), or bf16 if you have 30xx/40xx GPUs. Example strategy: cuda fp32 *1 -> cuda fp16
 
42
 
43
  NOTE: the new greedy tokenizer (https://github.com/BlinkDL/ChatRWKV/blob/main/tokenizer/rwkv_tokenizer.py) will tokenize '\n\n' as one single token instead of ['\n','\n']
44
 
 
59
 
60
  A good chat prompt (replace \n\n in xxx to \n):
61
  ```
62
+ User: hi
63
 
64
+ Assistant: Hi. I am your assistant and I will provide expert full response in full details. Please feel free to ask any question and I will always answer it.
65
 
66
+ User: xxx
67
 
68
+ Assistant:
69
  ```