Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ widget:
|
|
12 |
|
13 |
## Model description
|
14 |
|
15 |
-
This is the set of Chinese T5 models pre-trained by [UER-py](https://github.com/dbiir/UER-py/), which is introduced in [this paper](https://arxiv.org/abs/1909.05658).
|
16 |
|
17 |
The Text-to-Text Transfer Transformer (T5) leverages a unified text-to-text format and attains state-of-the-art results on a wide variety of English-language NLP tasks. Following their work, we released a series of Chinese T5 models.
|
18 |
|
@@ -120,6 +120,13 @@ python3 scripts/convert_t5_from_uer_to_huggingface.py --input_model_path cluecor
|
|
120 |
pages={241},
|
121 |
year={2019}
|
122 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
123 |
```
|
124 |
|
125 |
[small]:https://huggingface.co/uer/t5-small-chinese-cluecorpussmall
|
|
|
12 |
|
13 |
## Model description
|
14 |
|
15 |
+
This is the set of Chinese T5 models pre-trained by [UER-py](https://github.com/dbiir/UER-py/), which is introduced in [this paper](https://arxiv.org/abs/1909.05658). Besides, the models could also be pre-trained by [TencentPretrain](https://github.com/Tencent/TencentPretrain) introduced in [this paper](https://arxiv.org/abs/2212.06385), which inherits UER-py to support models with parameters above one billion, and extends it to a multimodal pre-training framework.
|
16 |
|
17 |
The Text-to-Text Transfer Transformer (T5) leverages a unified text-to-text format and attains state-of-the-art results on a wide variety of English-language NLP tasks. Following their work, we released a series of Chinese T5 models.
|
18 |
|
|
|
120 |
pages={241},
|
121 |
year={2019}
|
122 |
}
|
123 |
+
|
124 |
+
@article{zhao2023tencentpretrain,
|
125 |
+
title={TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities},
|
126 |
+
author={Zhao, Zhe and Li, Yudong and Hou, Cheng and Zhao, Jing and others},
|
127 |
+
journal={ACL 2023},
|
128 |
+
pages={217},
|
129 |
+
year={2023}
|
130 |
```
|
131 |
|
132 |
[small]:https://huggingface.co/uer/t5-small-chinese-cluecorpussmall
|