RaymondAISG
commited on
Commit
•
2bcc088
1
Parent(s):
8561bba
Update README.md
Browse files
README.md
CHANGED
@@ -20,7 +20,7 @@ SEA-LION stands for <i>Southeast Asian Languages In One Network</i>.
|
|
20 |
|
21 |
### Model Description
|
22 |
|
23 |
-
The SEA-
|
24 |
specifically trained to understand the SEA regional context.
|
25 |
|
26 |
For tokenization, the model employs the default tokenizer used in Meta-Llama-3-8B-Instruct.
|
@@ -35,7 +35,7 @@ The continued pre-training data for LLaMA3 8B SEA-LIONv2 base model encompasses
|
|
35 |
|
36 |
### Performance Benchmarks
|
37 |
|
38 |
-
SEA-
|
39 |
|
40 |
| Model | ARC | BBH | HellaSwag | MMLU | GSM8k | Average |
|
41 |
|----------------------|:-----:|:-----:|:---------:|:-----:|:------:|:-------:|
|
@@ -72,7 +72,7 @@ Note:
|
|
72 |
|
73 |
### Infrastructure
|
74 |
|
75 |
-
SEA-
|
76 |
on the following hardware:
|
77 |
|
78 |
| Training Details | LLaMA3 8B SEA-LIONv2 |
|
@@ -126,11 +126,13 @@ Wayne Lau<br>
|
|
126 |
Yeo Yeow Tong<br>
|
127 |
Yong Xianbin<br>
|
128 |
|
|
|
129 |
## Acknowledgements
|
130 |
|
131 |
AI Singapore is a national programme supported by the National Research Foundation, Singapore and hosted by the National University of Singapore.
|
132 |
Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore.
|
133 |
|
|
|
134 |
## Contact
|
135 |
|
136 |
For more info, please contact us using this [SEA-LION Inquiry Form](https://forms.gle/sLCUVb95wmGf43hi6)
|
|
|
20 |
|
21 |
### Model Description
|
22 |
|
23 |
+
The LLaMA3 8B SEA-LIONv model is a significant leap forward in the field of Natural Language Processing,
|
24 |
specifically trained to understand the SEA regional context.
|
25 |
|
26 |
For tokenization, the model employs the default tokenizer used in Meta-Llama-3-8B-Instruct.
|
|
|
35 |
|
36 |
### Performance Benchmarks
|
37 |
|
38 |
+
LLaMA3 8B SEA-LIONv has a similar English performance with LLaMA3-8B-Base model:
|
39 |
|
40 |
| Model | ARC | BBH | HellaSwag | MMLU | GSM8k | Average |
|
41 |
|----------------------|:-----:|:-----:|:---------:|:-----:|:------:|:-------:|
|
|
|
72 |
|
73 |
### Infrastructure
|
74 |
|
75 |
+
LLaMA3 8B SEA-LIONv2 was trained using [MosaicML Composer](https://github.com/mosaicml/composer)
|
76 |
on the following hardware:
|
77 |
|
78 |
| Training Details | LLaMA3 8B SEA-LIONv2 |
|
|
|
126 |
Yeo Yeow Tong<br>
|
127 |
Yong Xianbin<br>
|
128 |
|
129 |
+
|
130 |
## Acknowledgements
|
131 |
|
132 |
AI Singapore is a national programme supported by the National Research Foundation, Singapore and hosted by the National University of Singapore.
|
133 |
Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore.
|
134 |
|
135 |
+
|
136 |
## Contact
|
137 |
|
138 |
For more info, please contact us using this [SEA-LION Inquiry Form](https://forms.gle/sLCUVb95wmGf43hi6)
|