heegyu commited on
Commit
92fec50
1 Parent(s): 21e5011

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -0
README.md CHANGED
@@ -1,8 +1,38 @@
1
  ---
 
 
 
2
  datasets:
3
  - maywell/ko_Ultrafeedback_binarized
4
  - MrBananaHuman/kor_ethical_question_answer
 
 
 
 
 
5
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  ```
7
  from transformers import pipeline
8
 
 
1
  ---
2
+ license: mit
3
+ widget:
4
+ - text: "<user>:\n안녕하세요\n<bot>:\n안녕하세요! 오늘 날씨가 정말 좋아요 ^^ <|endoftext|>"
5
  datasets:
6
  - maywell/ko_Ultrafeedback_binarized
7
  - MrBananaHuman/kor_ethical_question_answer
8
+ language:
9
+ - ko
10
+ metrics:
11
+ - accuracy
12
+ pipeline_tag: text-classification
13
  ---
14
+
15
+ - Base Model: [42dot/42dot_LLM-SFT-1.3B](https://huggingface.co/42dot/42dot_LLM-SFT-1.3B)
16
+ - [maywell/ko_Ultrafeedback_binarized](https://huggingface.co/datasets/maywell/ko_Ultrafeedback_binarized)을 95:5 비율로 나뉜 뒤, 95%를 학습에 사용하고 5%는 평가에 사용
17
+ - Test 정확도: 74.38
18
+ ```
19
+ load_dataset("maywell/ko_Ultrafeedback_binarized", split="train").train_test_split(0.05, seed=42)
20
+ ```
21
+ - [MrBananaHuman/kor_ethical_question_answer](https://huggingface.co/datasets/MrBananaHuman/kor_ethical_question_answer) 중 1000개를 test set으로 사용
22
+ - Test 정확도: 100 (문제가 쉬운 것 같습니다)
23
+
24
+ ## Hyperparameters:
25
+ - Batch: 128
26
+ - Learning Rate: 1e-5 -> 1e-6 (Linear Decay)
27
+ - Optimizer: AdamW (beta1 = 0.9, beta2 = 0.999)
28
+ - Epoch: 3 (main revision은 1 epoch)
29
+
30
+ ## Usage
31
+ - 기존 42dot SFT 모델의 대화 템플릿을 사용.
32
+ - 사용자의 발화는 `<user>:\n`로 시작
33
+ - Bot의 발화는 `<bot>:\n`으로 시작
34
+
35
+
36
  ```
37
  from transformers import pipeline
38