dreamerdeo
commited on
Commit
•
92c0fe6
1
Parent(s):
6df499d
Update README.md
Browse files
README.md
CHANGED
@@ -52,21 +52,26 @@ These models, released under the **Apache 2.0 license**, provide enhanced access
|
|
52 |
Sailor2 builds upon the foundation of the awesome multilingual model [Qwen 2.5](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and
|
53 |
is continuously pre-trained on **500B tokens** to support **15 languages** better with a unified model.
|
54 |
These languages include English, Chinese, Burmese, Cebuano, Ilocano, Indonesian, Javanese, Khmer, Lao, Malay, Sundanese, Tagalog, Thai, Vietnamese, and Waray.
|
55 |
-
By addressing the growing demand for diverse, robust, and accessible language models,
|
56 |
-
Sailor2
|
57 |
-
|
58 |
-
Refer to [Sailor2 Website](https://sea-sailor.github.io/blog/sailor2/) for more training details.
|
59 |
|
60 |
## Model Summary
|
61 |
- **Model Collections:** [Base Model & Chat Model](https://huggingface.co/collections/sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b)
|
62 |
-
- **Project Website:** [
|
63 |
- **Codebase:** [github.com/sail-sg/sailor2](https://github.com/sail-sg/sailor2)
|
64 |
- **Technical Report:** Coming Soon
|
65 |
|
66 |
|
67 |
-
|
68 |
## Training details
|
69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
70 |
|
71 |
## Requirements
|
72 |
The code of Sailor2 has been in the latest Hugging face transformers and we advise you to install `transformers==4.46.3`.
|
|
|
52 |
Sailor2 builds upon the foundation of the awesome multilingual model [Qwen 2.5](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and
|
53 |
is continuously pre-trained on **500B tokens** to support **15 languages** better with a unified model.
|
54 |
These languages include English, Chinese, Burmese, Cebuano, Ilocano, Indonesian, Javanese, Khmer, Lao, Malay, Sundanese, Tagalog, Thai, Vietnamese, and Waray.
|
55 |
+
By addressing the growing demand for diverse, robust, and accessible language models, Sailor2 seeks to serve the underserved in SEA areas with open, inclusive, and accessible multilingual LLMs.
|
56 |
+
The Sailor2 model comes in three sizes, 1B, 8B, and 20B, which are **expanded from the Qwen2.5 base models** of 0.5B, 7B, and 14B, respectively.
|
|
|
|
|
57 |
|
58 |
## Model Summary
|
59 |
- **Model Collections:** [Base Model & Chat Model](https://huggingface.co/collections/sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b)
|
60 |
+
- **Project Website:** [sailorllm.github.io/blog/sailor2](https://sailorllm.github.io/blog/sailor2)
|
61 |
- **Codebase:** [github.com/sail-sg/sailor2](https://github.com/sail-sg/sailor2)
|
62 |
- **Technical Report:** Coming Soon
|
63 |
|
64 |
|
|
|
65 |
## Training details
|
66 |
|
67 |
+
During development, we employ a range of advanced technologies to ensure top-tier performance and efficiency:
|
68 |
+
|
69 |
+
1. model expansion
|
70 |
+
2. optimized data mixing strategies
|
71 |
+
3. multi-stage pre-training protocols
|
72 |
+
4. advanced multilingual post-training
|
73 |
+
|
74 |
+
|
75 |
|
76 |
## Requirements
|
77 |
The code of Sailor2 has been in the latest Hugging face transformers and we advise you to install `transformers==4.46.3`.
|