Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,7 @@ pipeline_tag: text-generation
|
|
24 |
|
25 |
**DNA 1.0 8B Instruct** is a <u>state-of-the-art (**SOTA**)</u> bilingual language model based on Llama architecture, specifically optimized for Korean language understanding and generation, while also maintaining strong English capabilities. The model was developed through a sophisticated process involving model merging via spherical linear interpolation (**SLERP**) with Llama 3.1 8B Instruct, and underwent knowledge distillation (**KD**) using Llama 3.1 405B as the teacher model. It was extensively trained through continual pre-training (**CPT**) with a high-quality Korean dataset. The training pipeline was completed with supervised fine-tuning (**SFT**) and direct preference optimization (**DPO**) to align with human preferences and enhance instruction-following abilities.
|
26 |
|
27 |
-
DNA 1.0 8B Instruct was fine-tuned on approximately
|
28 |
|
29 |
- **Developed by:** Dnotitia Inc.
|
30 |
- **Supported Languages:** Korean, English
|
|
|
24 |
|
25 |
**DNA 1.0 8B Instruct** is a <u>state-of-the-art (**SOTA**)</u> bilingual language model based on Llama architecture, specifically optimized for Korean language understanding and generation, while also maintaining strong English capabilities. The model was developed through a sophisticated process involving model merging via spherical linear interpolation (**SLERP**) with Llama 3.1 8B Instruct, and underwent knowledge distillation (**KD**) using Llama 3.1 405B as the teacher model. It was extensively trained through continual pre-training (**CPT**) with a high-quality Korean dataset. The training pipeline was completed with supervised fine-tuning (**SFT**) and direct preference optimization (**DPO**) to align with human preferences and enhance instruction-following abilities.
|
26 |
|
27 |
+
DNA 1.0 8B Instruct was fine-tuned on approximately 7B tokens of carefully curated data and has undergone extensive instruction tuning to enhance its ability to follow complex instructions and engage in natural conversations.
|
28 |
|
29 |
- **Developed by:** Dnotitia Inc.
|
30 |
- **Supported Languages:** Korean, English
|