wonseok commited on
Commit
6b69fc2
ยท
1 Parent(s): e90fbd2

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +42 -0
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## How to use
2
+ ```python
3
+ import transformers
4
+
5
+ model = transformers.GPT2LMHeadModel.from_pretrained("lbox/lcube-base")
6
+ tokenizer = transformers.AutoTokenizer.from_pretrained(
7
+ "lbox/lcube-base",
8
+ bos_token="[BOS]",
9
+ unk_token="[UNK]",
10
+ pad_token="[PAD]",
11
+ mask_token="[MASK]",
12
+ )
13
+
14
+ text = "ํ”ผ๊ณ ์ธ์€ ๋ถˆ์ƒ์ง€์— ์žˆ๋Š” ์ปคํ”ผ์ˆ์—์„œ, ํ”ผํ•ด์ž B์œผ๋กœ๋ถ€ํ„ฐ"
15
+ model_inputs = tokenizer(text,
16
+ max_length=1024,
17
+ padding=True,
18
+ truncation=True,
19
+ return_tensors='pt')
20
+ out = model.generate(
21
+ model_inputs["input_ids"],
22
+ max_new_tokens=150,
23
+ pad_token_id=tokenizer.pad_token_id,
24
+ use_cache=True,
25
+ repetition_penalty=1.2,
26
+ top_k=5,
27
+ top_p=0.9,
28
+ temperature=1,
29
+ num_beams=2,
30
+ )
31
+ tokenizer.batch_decode(out)
32
+ ```
33
+
34
+ For more information please visit <https://github.com/lbox-kr/lbox_open>.
35
+
36
+
37
+
38
+ ## Licensing Information
39
+
40
+ Copyright 2022-present LBox Co. Ltd.
41
+
42
+ Licensed under the CC BY-NC-ND 4.0