dreamerdeo commited on
Commit
ca6a4fe
1 Parent(s): 60ff0c3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -6
README.md CHANGED
@@ -48,24 +48,28 @@ Our research highlights a strong demand for models in the **8B and 20B parameter
48
  such as speculative decoding and research purposes.
49
  These models, released under the **Apache 2.0 license**, provide enhanced accessibility to advanced language technologies across the region.
50
 
51
-
52
  Sailor2 builds upon the foundation of the awesome multilingual model [Qwen 2.5](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and
53
  is continuously pre-trained on **500B tokens** to support **15 languages** better with a unified model.
54
  These languages include English, Chinese, Burmese, Cebuano, Ilocano, Indonesian, Javanese, Khmer, Lao, Malay, Sundanese, Tagalog, Thai, Vietnamese, and Waray.
55
- By addressing the growing demand for diverse, robust, and accessible language models,
56
- Sailor2 seeks to serve the underserved in SEA areas with open, inclusive, and accessible multilingual LLMs.
57
-
58
- Refer to [Sailor2 Website](https://sea-sailor.github.io/blog/sailor2/) for more training details.
59
 
60
  ## Model Summary
61
  - **Model Collections:** [Base Model & Chat Model](https://huggingface.co/collections/sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b)
62
- - **Project Website:** [sea-sailor.github.io/blog/sailor2](https://sea-sailor.github.io/blog/sailor2/)
63
  - **Codebase:** [github.com/sail-sg/sailor2](https://github.com/sail-sg/sailor2)
64
  - **Technical Report:** Coming Soon
65
 
66
 
67
  ## Training details
68
 
 
 
 
 
 
 
 
69
 
70
  ## Requirements
71
  The code of Sailor2 has been in the latest Hugging face transformers and we advise you to install `transformers==4.46.3`.
 
48
  such as speculative decoding and research purposes.
49
  These models, released under the **Apache 2.0 license**, provide enhanced accessibility to advanced language technologies across the region.
50
 
 
51
  Sailor2 builds upon the foundation of the awesome multilingual model [Qwen 2.5](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and
52
  is continuously pre-trained on **500B tokens** to support **15 languages** better with a unified model.
53
  These languages include English, Chinese, Burmese, Cebuano, Ilocano, Indonesian, Javanese, Khmer, Lao, Malay, Sundanese, Tagalog, Thai, Vietnamese, and Waray.
54
+ By addressing the growing demand for diverse, robust, and accessible language models, Sailor2 seeks to serve the underserved in SEA areas with open, inclusive, and accessible multilingual LLMs.
55
+ The Sailor2 model comes in three sizes, 1B, 8B, and 20B, which are **expanded from the Qwen2.5 base models** of 0.5B, 7B, and 14B, respectively.
 
 
56
 
57
  ## Model Summary
58
  - **Model Collections:** [Base Model & Chat Model](https://huggingface.co/collections/sail/sailor2-language-models-674d7c9e6b4dbbd9a869906b)
59
+ - **Project Website:** [sailorllm.github.io/blog/sailor2](https://sailorllm.github.io/blog/sailor2)
60
  - **Codebase:** [github.com/sail-sg/sailor2](https://github.com/sail-sg/sailor2)
61
  - **Technical Report:** Coming Soon
62
 
63
 
64
  ## Training details
65
 
66
+ During development, we employ a range of advanced technologies to ensure top-tier performance and efficiency:
67
+
68
+ 1. model expansion
69
+ 2. optimized data mixing strategies
70
+ 3. multi-stage pre-training protocols
71
+ 4. advanced multilingual post-training
72
+
73
 
74
  ## Requirements
75
  The code of Sailor2 has been in the latest Hugging face transformers and we advise you to install `transformers==4.46.3`.