add readme
Browse files
README.md
ADDED
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
## Usage
|
2 |
+
Please use BertTokenizerFast as tokenizer instead of AutoTokenizer.
|
3 |
+
請使用 BertTokenizerFast 而非 AutoTokenizer。
|
4 |
+
```
|
5 |
+
from transformers import (
|
6 |
+
BertTokenizerFast,
|
7 |
+
AutoModelForCausalLM,
|
8 |
+
)
|
9 |
+
|
10 |
+
tokenizer = BertTokenizerFast.from_pretrained('bert-base-chinese')
|
11 |
+
model = AutoModelForCausalLM.from_pretrained('ckiplab/gpt2-base-chinese')
|
12 |
+
```
|
13 |
+
### Input Format
|
14 |
+
```
|
15 |
+
C' = [c1, c2, ..., [HL], a1, ..., a|A|, [HL], ..., c|C|]
|
16 |
+
```
|
17 |
+
### Input Example
|
18 |
+
```
|
19 |
+
哈利·波特是英國作家[HL]羅琳[HL]撰寫的七部幻想小說系列。
|
20 |
+
```
|
21 |
+
> 誰撰寫哈利·波特?
|