dreamerdeo commited on
Commit
92c0fe6
1 Parent(s): 6df499d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -6
README.md CHANGED
@@ -52,21 +52,26 @@ These models, released under the **Apache 2.0 license**, provide enhanced access
52
  Sailor2 builds upon the foundation of the awesome multilingual model [Qwen 2.5](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and
53
  is continuously pre-trained on **500B tokens** to support **15 languages** better with a unified model.
54
  These languages include English, Chinese, Burmese, Cebuano, Ilocano, Indonesian, Javanese, Khmer, Lao, Malay, Sundanese, Tagalog, Thai, Vietnamese, and Waray.
55
- By addressing the growing demand for diverse, robust, and accessible language models,
56
- Sailor2 seeks to serve the underserved in SEA areas with open, inclusive, and accessible multilingual LLMs.
57
-
58
- Refer to [Sailor2 Website](https://sea-sailor.github.io/blog/sailor2/) for more training details.
59
 
60
  ## Model Summary
61
  - **Model Collections:** [Base Model & Chat Model](https://huggingface.co/collections/sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b)
62
- - **Project Website:** [sea-sailor.github.io/blog/sailor2](https://sea-sailor.github.io/blog/sailor2/)
63
  - **Codebase:** [github.com/sail-sg/sailor2](https://github.com/sail-sg/sailor2)
64
  - **Technical Report:** Coming Soon
65
 
66
 
67
-
68
  ## Training details
69
 
 
 
 
 
 
 
 
 
70
 
71
  ## Requirements
72
  The code of Sailor2 has been in the latest Hugging face transformers and we advise you to install `transformers==4.46.3`.
 
52
  Sailor2 builds upon the foundation of the awesome multilingual model [Qwen 2.5](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and
53
  is continuously pre-trained on **500B tokens** to support **15 languages** better with a unified model.
54
  These languages include English, Chinese, Burmese, Cebuano, Ilocano, Indonesian, Javanese, Khmer, Lao, Malay, Sundanese, Tagalog, Thai, Vietnamese, and Waray.
55
+ By addressing the growing demand for diverse, robust, and accessible language models, Sailor2 seeks to serve the underserved in SEA areas with open, inclusive, and accessible multilingual LLMs.
56
+ The Sailor2 model comes in three sizes, 1B, 8B, and 20B, which are **expanded from the Qwen2.5 base models** of 0.5B, 7B, and 14B, respectively.
 
 
57
 
58
  ## Model Summary
59
  - **Model Collections:** [Base Model & Chat Model](https://huggingface.co/collections/sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b)
60
+ - **Project Website:** [sailorllm.github.io/blog/sailor2](https://sailorllm.github.io/blog/sailor2)
61
  - **Codebase:** [github.com/sail-sg/sailor2](https://github.com/sail-sg/sailor2)
62
  - **Technical Report:** Coming Soon
63
 
64
 
 
65
  ## Training details
66
 
67
+ During development, we employ a range of advanced technologies to ensure top-tier performance and efficiency:
68
+
69
+ 1. model expansion
70
+ 2. optimized data mixing strategies
71
+ 3. multi-stage pre-training protocols
72
+ 4. advanced multilingual post-training
73
+
74
+
75
 
76
  ## Requirements
77
  The code of Sailor2 has been in the latest Hugging face transformers and we advise you to install `transformers==4.46.3`.