|
--- |
|
language: |
|
- zh |
|
--- |
|
|
|
# COIG-Kun Label Model |
|
|
|
## Model Details |
|
- **Name:** COIG-Kun Label Model |
|
- **Release Date:** 2023.12.04 |
|
- **Github URL:** [COIG-Kun](https://github.com/Zheng0428/COIG-Kun) |
|
- **Developers:** Tianyu Zheng*, Shuyue Guo*, Xingwei Qu, Xinrun Du, Wenhu Chen, Jie Fu, Wenhao Huang, Ge Zhang |
|
|
|
## Model Description |
|
The Label Model is a part of the Kun project, which aims to enhance language model training through a novel data augmentation paradigm, leveraging principles of self-alignment and instruction backtranslation. The model is specifically fine-tuned to generate high-quality instructional data, a critical component in the project's approach to data augmentation and language model training. |
|
|
|
## Intended Use |
|
- **Primary Use:** The Label Model is designed for generating instructional data to fine-tune language models. |
|
- **Target Users:** Researchers and developers in NLP and ML, particularly those working on language model training and data augmentation. |
|
|
|
## Training Data |
|
The Label Model is trained using approximately ten thousand high-quality seed instructions. These instructions were meticulously curated to ensure the effectiveness of the training process and to produce high-quality outputs for use as instructional data. |
|
|
|
## Training Process |
|
- **Base Model:** Yi-34B |
|
- **Epochs:** 6 |
|
- **Learning Rate:** 1e-5 |
|
- **Fine-Tuning Method:** The model was fine-tuned on high-quality seed instructions, with the responses to these instructions used as outputs and the instructions themselves as inputs. |
|
|
|
## Evaluation |
|
The Label Model was evaluated on its ability to generate high-quality instructional data, focusing on the relevancy, clarity, and usability of the instructions for language model training. |
|
|
|
|
|
## Ethical Considerations |
|
- Users should be aware of potential biases in the training data, which could be reflected in the model's outputs. |
|
- The model should not be used for generating harmful or misleading content. |
|
|
|
## Citing the Model |
|
To cite the Label Model in academic work, please use the following reference: |
|
|
|
```bibtex |
|
@misc{COIG-Kun, |
|
title={Kun: Answer Polishment Saves Your Time for Using Intruction Backtranslation on Self-Alignment}, |
|
author={Tianyu, Zheng* and Shuyue, Guo* and Xingwei, Qu and Xinrun, Du and Wenhu, Chen and Jie, Fu and Wenhao, Huang and Ge, Zhang}, |
|
year={2023}, |
|
publisher={GitHub}, |
|
journal={GitHub repository}, |
|
howpublished={https://github.com/Zheng0428/COIG-Kun} |
|
} |
|
``` |
