Update README.md
Browse files
README.md
CHANGED
@@ -7,6 +7,86 @@ language:
|
|
7 |
library_name: adapter-transformers
|
8 |
---
|
9 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
# Model Card for K23 MiniMed
|
11 |
|
12 |
This is a Mistral 7b Beta Medical Fine Tune with a short number of steps , inspired by [Wonhyeong Seo](https://www.huggingface.co/wseo) great mentorship during Krew x Huggingface 2023 hackathon.
|
|
|
7 |
library_name: adapter-transformers
|
8 |
---
|
9 |
|
10 |
+
# K23 MiniMed ๋ชจ๋ธ ์นด๋
|
11 |
+
|
12 |
+
K23 MiniMed๋ Krew x Huggingface 2023 ํด์ปคํค์์ ์ํ์ ๋ฉํ ์ ์ง๋ํ์ ๊ฐ๋ฐ๋ Mistral 7b Beta Medical Fine Tune ๋ชจ๋ธ์
๋๋ค.
|
13 |
+
|
14 |
+
## ๋ชจ๋ธ ์ธ๋ถ์ฌํญ
|
15 |
+
|
16 |
+
- **๊ฐ๋ฐ์:** [Tonic](https://huggingface.co/Tonic)
|
17 |
+
|
18 |
+
- **ํ์:** [Tonic](https://huggingface.co/Tonic)
|
19 |
+
- **๊ณต์ ์:** K23-Krew-Hackathon
|
20 |
+
- **๋ชจ๋ธ ์ ํ:** Mistral 7B-Beta Medical Fine Tune
|
21 |
+
- **์ธ์ด (NLP):** ์์ด
|
22 |
+
- **๋ผ์ด์ผ์ค:** MIT
|
23 |
+
- **Fine-tuning ๊ธฐ๋ฐ ๋ชจ๋ธ:** [Zephyr 7B-Beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
|
24 |
+
|
25 |
+
### ๋ชจ๋ธ ์ถ์ฒ
|
26 |
+
- **์ ์ฅ์:** [github](https://github.com/Josephrp/AI-challenge-hackathon/blob/master/mistral7b-beta_finetune.ipynb)
|
27 |
+
- **๋ฐ๋ชจ:** [pseudolab/K23MiniMed](https://huggingface.co/spaces/pseudolab/K23MiniMed)
|
28 |
+
## ์ฌ์ฉ๋ฒ
|
29 |
+
์ด ๋ชจ๋ธ์ ๊ต์ก ๋ชฉ์ ์ผ๋ก๋ง ์ํ ์ง๋ฌธ ๋ต๋ณ์ ์ํ ๋ํํ ์ ํ๋ฆฌ์ผ์ด์
์ฉ์
๋๋ค.
|
30 |
+
### ์ง์ ์ฌ์ฉ
|
31 |
+
|
32 |
+
Gradio ์ฑ๋ด ์ฑ์ ๋ง๋ค์ด ์ํ์ ์ง๋ฌธ์ ํ๊ณ ๋ํ์์ผ๋ก ๋ต๋ณ์ ๋ฐ์ต๋๋ค.
|
33 |
+
|
34 |
+
### ํ๋ฅ ์ฌ์ฉ
|
35 |
+
|
36 |
+
์ด ๋ชจ๋ธ์ ๊ต์ก์ฉ์ผ๋ก๋ง ์ฌ์ฉ๋ฉ๋๋ค. ์ถ๊ฐ์ ์ธ Fine-tuning๊ณผ ์ฌ์ฉ ์์๋ก๋ ๊ณต์ค ๋ณด๊ฑด & ์์, ๊ฐ์ธ ๋ณด๊ฑด & ์์, ์ํ Q & A๊ฐ ์์ต๋๋ค.
|
37 |
+
|
38 |
+
### ์ถ์ฒ์ฌํญ
|
39 |
+
|
40 |
+
์ฌ์ฉ ์ ์ ํญ์ ์ด ๋ชจ๋ธ์ ํ๊ฐํ๊ณ ๋ฒค์น๋งํนํ์ญ์์ค. ์ฌ์ฉ ์ ์ ํธํฅ์ ํ๊ฐํ์ญ์์ค. ๊ทธ๋๋ก ์ฌ์ฉํ์ง ๋ง์๊ณ ์ถ๊ฐ์ ์ผ๋ก Fine-tuningํ์ญ์์ค.
|
41 |
+
|
42 |
+
## ํ๋ จ ์ธ๋ถ์ฌํญ
|
43 |
+
|
44 |
+
๋ชจ๋ธ์ ํ๋ จ ์์ค์ ๋ค์๊ณผ ๊ฐ์ต๋๋ค:
|
45 |
+
| ๋จ๊ณ | ํ๋ จ ์์ค |
|
46 |
+
|------|--------------|
|
47 |
+
| 50 | 0.993800 |
|
48 |
+
| 100 | 0.620600 |
|
49 |
+
| 150 | 0.547100 |
|
50 |
+
| 200 | 0.524100 |
|
51 |
+
| 250 | 0.520500 |
|
52 |
+
| 300 | 0.559800 |
|
53 |
+
| 350 | 0.535500 |
|
54 |
+
| 400 | 0.505400 |
|
55 |
+
### ํ๋ จ ๋ฐ์ดํฐ
|
56 |
+
๋ชจ๋ธ์ ํ์ต ๊ฐ๋ฅํ ๋งค๊ฐ๋ณ์: 21260288, ๋ชจ๋ ๋งค๊ฐ๋ณ์: 3773331456, ํ์ต ๊ฐ๋ฅํ %: 0.5634354746703705.
|
57 |
+
|
58 |
+
### ๊ฒฐ๊ณผ
|
59 |
+
|
60 |
+
global_step=400์์์ ํ๋ จ ์์ค์ 0.6008514881134033์
๋๋ค.
|
61 |
+
|
62 |
+
## ํ๊ฒฝ ์ํฅ
|
63 |
+
|
64 |
+
๋ชจ๋ธ์ ํ๊ฒฝ ์ํฅ์ ๋จธ์ ๋ฌ๋ ์ํฅ ๊ณ์ฐ๊ธฐ๋ฅผ ์ฌ์ฉํ์ฌ ๊ณ์ฐํ ์ ์์ต๋๋ค. ์ถ์ ์ ์ ๊ณตํ๊ธฐ ์ํด์๋ ๋ ๋ง์ ์ธ๋ถ ์ ๋ณด๊ฐ ํ์ํฉ๋๋ค.
|
65 |
+
|
66 |
+
## ๊ธฐ์ ์ฌ์
|
67 |
+
|
68 |
+
### ๋ชจ๋ธ ์ํคํ
์ฒ์ ๋ชฉํ
|
69 |
+
|
70 |
+
๋ชจ๋ธ์ ํน์ ์ค์ ์ ๊ฐ์ง PeftModelForCausalLM์ ์ฌ์ฉํฉ๋๋ค.
|
71 |
+
|
72 |
+
### ์ปดํจํ
์ธํ๋ผ
|
73 |
+
|
74 |
+
#### ํ๋์จ์ด
|
75 |
+
|
76 |
+
๋ชจ๋ธ์ A100 ํ๋์จ์ด์์ ํ๋ จ๋์์ต๋๋ค.
|
77 |
+
|
78 |
+
#### ์ํํธ์จ์ด
|
79 |
+
|
80 |
+
์ฌ์ฉ๋ ์ํํธ์จ์ด์๋ peft, torch, bitsandbytes, python, ๊ทธ๋ฆฌ๊ณ huggingface๊ฐ ํฌํจ๋ฉ๋๋ค.
|
81 |
+
|
82 |
+
## ๋ชจ๋ธ ์นด๋ ์์ฑ์
|
83 |
+
|
84 |
+
[Tonic](https://huggingface.co/Tonic)
|
85 |
+
|
86 |
+
## ๋ชจ๋ธ ์นด๋ ์ฐ๋ฝ์ฒ
|
87 |
+
|
88 |
+
[Tonic](https://huggingface.co/Tonic)
|
89 |
+
|
90 |
# Model Card for K23 MiniMed
|
91 |
|
92 |
This is a Mistral 7b Beta Medical Fine Tune with a short number of steps , inspired by [Wonhyeong Seo](https://www.huggingface.co/wseo) great mentorship during Krew x Huggingface 2023 hackathon.
|