KaeriJenti commited on
Commit
117dab7
·
1 Parent(s): b823736

Update README.md

Browse files

This Model was Finetuned By Kaeri and Jenti. We did not use GSM8k samples when generating data.
Also we were careful of data contamination by similarity filtering
the training data if the data correspond to any of the following list.

Files changed (1) hide show
  1. README.md +43 -0
README.md CHANGED
@@ -1,3 +1,46 @@
1
  ---
2
  license: llama2
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama2
3
  ---
4
+
5
+ <h1>kaori-34b-v3 Model Card</h1>
6
+
7
+ This Model was Finetuned By Kaeri and Jenti.
8
+
9
+ <h3>Datasets Strategy</h3>
10
+
11
+ - Open-Platypus
12
+ - Dolphin
13
+
14
+ We trained the model with 100% Open-Platypus data and 5% Dolphin data and applied SFT strategy.
15
+
16
+ We did not use GSM8k samples when generating data.
17
+ Also we were careful of data contamination by similarity filtering
18
+ the training data if the data correspond to any of the following list.
19
+
20
+ <pre>
21
+ filtering_tasks = [
22
+ 'cot_gsm8k',
23
+ 'cot_gsm8k_ii',
24
+ 'drop:2.0.0',
25
+ 'winogrande:1.1.0'
26
+ 'task228_arc_answer_generation_easy',
27
+ 'ai2_arc/ARC-Challenge:1.0.0',
28
+ 'ai2_arc/ARC-Easy:1.0.0',
29
+ 'task229_arc_answer_generation_hard',
30
+ 'hellaswag:1.1.0',
31
+ 'task1389_hellaswag_completion'
32
+ ]
33
+ </pre>
34
+
35
+
36
+ <h3>Framework:</h3>
37
+
38
+ - https://github.com/hiyouga/LLaMA-Factory
39
+
40
+
41
+ <h3>Parameters:</h3>
42
+
43
+ - Finetune_Type : LoRA
44
+ - GPUs : A100x4(80GB)
45
+ - Epochs : 3
46
+ - Batchsize : 8