Update README.md
Browse files
README.md
CHANGED
|
@@ -49,6 +49,7 @@ During the incremental training process, we used 160 A100s with a total of 40GB
|
|
| 49 |
Throughout the training process, we encountered various issues such as machine crashes, underlying framework bugs, and loss spikes. However, we ensured the stability of the incremental training by making rapid adjustments. We have also released the loss curve during the training process to help everyone understand the potential issues that may arise.
|
| 50 |
|
| 51 |
<img src="https://huggingface.co/datasets/suolyer/testb/resolve/main/loss.png" width=1000 height=600>
|
|
|
|
| 52 |
### 多任务有监督微调 Supervised finetuning
|
| 53 |
|
| 54 |
在多任务有监督微调阶段，采用了课程学习（curiculum learning）和增量训练（continual learning）的策略，用大模型辅助划分已有的数据难度，然后通过“Easy To Hard”的方式，分多个阶段进行SFT训练。
|
|
@@ -76,7 +77,7 @@ We implemented the HFT training process on an internally developed framework, wh
|
|
| 76 |
|
| 77 |
### 效果评估 Performance
|
| 78 |
|
| 79 |
-
|
| 80 |
|
| 81 |
## 使用 Usage
|
| 82 |
|
|
|
|
| 49 |
Throughout the training process, we encountered various issues such as machine crashes, underlying framework bugs, and loss spikes. However, we ensured the stability of the incremental training by making rapid adjustments. We have also released the loss curve during the training process to help everyone understand the potential issues that may arise.
|
| 50 |
|
| 51 |
<img src="https://huggingface.co/datasets/suolyer/testb/resolve/main/loss.png" width=1000 height=600>
|
| 52 |
+
|
| 53 |
### 多任务有监督微调 Supervised finetuning
|
| 54 |
|
| 55 |
在多任务有监督微调阶段，采用了课程学习（curiculum learning）和增量训练（continual learning）的策略，用大模型辅助划分已有的数据难度，然后通过“Easy To Hard”的方式，分多个阶段进行SFT训练。
|
|
|
|
| 77 |
|
| 78 |
### 效果评估 Performance
|
| 79 |
|
| 80 |
+
<img src="" width=1000 height=600>
|
| 81 |
|
| 82 |
## 使用 Usage
|
| 83 |
|