Commit
·
117dab7
1
Parent(s):
b823736
Update README.md
Browse filesThis Model was Finetuned By Kaeri and Jenti. We did not use GSM8k samples when generating data.
Also we were careful of data contamination by similarity filtering
the training data if the data correspond to any of the following list.
README.md
CHANGED
@@ -1,3 +1,46 @@
|
|
1 |
---
|
2 |
license: llama2
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: llama2
|
3 |
---
|
4 |
+
|
5 |
+
<h1>kaori-34b-v3 Model Card</h1>
|
6 |
+
|
7 |
+
This Model was Finetuned By Kaeri and Jenti.
|
8 |
+
|
9 |
+
<h3>Datasets Strategy</h3>
|
10 |
+
|
11 |
+
- Open-Platypus
|
12 |
+
- Dolphin
|
13 |
+
|
14 |
+
We trained the model with 100% Open-Platypus data and 5% Dolphin data and applied SFT strategy.
|
15 |
+
|
16 |
+
We did not use GSM8k samples when generating data.
|
17 |
+
Also we were careful of data contamination by similarity filtering
|
18 |
+
the training data if the data correspond to any of the following list.
|
19 |
+
|
20 |
+
<pre>
|
21 |
+
filtering_tasks = [
|
22 |
+
'cot_gsm8k',
|
23 |
+
'cot_gsm8k_ii',
|
24 |
+
'drop:2.0.0',
|
25 |
+
'winogrande:1.1.0'
|
26 |
+
'task228_arc_answer_generation_easy',
|
27 |
+
'ai2_arc/ARC-Challenge:1.0.0',
|
28 |
+
'ai2_arc/ARC-Easy:1.0.0',
|
29 |
+
'task229_arc_answer_generation_hard',
|
30 |
+
'hellaswag:1.1.0',
|
31 |
+
'task1389_hellaswag_completion'
|
32 |
+
]
|
33 |
+
</pre>
|
34 |
+
|
35 |
+
|
36 |
+
<h3>Framework:</h3>
|
37 |
+
|
38 |
+
- https://github.com/hiyouga/LLaMA-Factory
|
39 |
+
|
40 |
+
|
41 |
+
<h3>Parameters:</h3>
|
42 |
+
|
43 |
+
- Finetune_Type : LoRA
|
44 |
+
- GPUs : A100x4(80GB)
|
45 |
+
- Epochs : 3
|
46 |
+
- Batchsize : 8
|