squarelike's picture
Create README.md
ec6c9e9
|
raw
history blame
1.29 kB
metadata
language:
  - ko
tags:
  - pytorch
  - causal-lm
  - medical
license: apache-2.0
pipeline_tag: text-generation

Llama-Ko-Medical-7b

llama2-ko-medical์€ llama-2-ko๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ์˜๋ฃŒ ๋ถ„์•ผ์˜ ํ•œ๊ธ€ raw ๋ฐ์ดํ„ฐ๋ฅผ ํ•™์Šต์‹œํ‚จ ๊ธฐ๋ฐ˜ ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.

ํ•™์Šต ๋ฐ์ดํ„ฐ

polyglot-ko-medical์€ ์•ฝ 526MB์˜ ์˜๋ฃŒ ๋ถ„์•ผ ํ•œ๊ธ€ ๋ง๋ญ‰์น˜๋กœ ํ•™์Šต๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ์ฃผ์š” ๋ฐ์ดํ„ฐ์…‹์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

Source Size (MB) Link
AIHub ์˜๋ฃŒ, ๋ฒ•๋ฅ  ์ „๋ฌธ ์„œ์  ๋ง๋ญ‰์น˜ 351.0 aihub.or.kr
ํ•˜์ด๋‹ฅ ๋‰ด์Šค ๋ฐ์ดํ„ฐ 97.3 https://www.hidoc.co.kr/
AIHub ์ „๋ฌธ๋ถ„์•ผ ํ•œ์˜ ๋ง๋ญ‰์น˜ 63.4 aihub.or.kr
์งˆ๋ณ‘๊ด€๋ฆฌ์ฒญ ๊ตญ๊ฐ€๊ฑด๊ฐ•์ •๋ณดํฌํ„ธ 8.33 health.kdca.go.kr
๋ณด๊ฑด๋ณต์ง€๋ถ€ ๊ตญ๊ฐ€์ •์‹ ๊ฑด๊ฐ•์ •๋ณดํฌํ„ธ < 1.0 mentalhealth.go.kr

ํ•™์Šต

llama2-ko-medical-7b๋Š” llama-2-ko์—์„œ qlora๋กœ ์ถ”๊ฐ€ ํ•™์Šต๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

  • lora_alpha: 16
  • lora_dropout: 0.01
  • lora_r: 64
  • target_modules: q_proj, v_proj
  • epoch: 3
  • learning_rate: 3e-4