Update README.md
Browse files
README.md
CHANGED
@@ -37,6 +37,14 @@ inference: true
|
|
37 |
### 🎻 Welcome!
|
38 |
Sasando-1 is a tiny, highly experimental Indonesian text generator built using the Phi-3 architecture. It comes with two variations of microscopic sizes: 7M and 25M parameters. It is trained on a tightly-controlled Indo4B dataset filtered to only have 18000 unique words. The method is inspired by Microsoft's TinyStories paper which demonstrates that a tiny language model can produce fluent text when trained on tightly-controlled dataset.
|
39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
### ✨ Specs
|
41 |
- Comes with 7M and 25M parameters
|
42 |
- Based on Phi-3 architecture
|
|
|
37 |
### 🎻 Welcome!
|
38 |
Sasando-1 is a tiny, highly experimental Indonesian text generator built using the Phi-3 architecture. It comes with two variations of microscopic sizes: 7M and 25M parameters. It is trained on a tightly-controlled Indo4B dataset filtered to only have 18000 unique words. The method is inspired by Microsoft's TinyStories paper which demonstrates that a tiny language model can produce fluent text when trained on tightly-controlled dataset.
|
39 |
|
40 |
+
### 🇮🇩 Context
|
41 |
+
Indonesia has +700 languages, and many of them are dying at an alarming rate. Language technologies like generative AI can play a massive role in language preservation. However, Indonesia has several contextual issues:
|
42 |
+
|
43 |
+
- Many languages, including those with millions of speakers, have low-volume digital resources
|
44 |
+
- Running large models can be costly, while Indonesia is a middle-income country with little funding
|
45 |
+
|
46 |
+
Overcoming these challenges require developers to work with what little data and money that they have. Sasando-1 is a prototypical demonstration that thinly-available resources can potentially still be leveraged to develop generative models with cheap compute.
|
47 |
+
|
48 |
### ✨ Specs
|
49 |
- Comes with 7M and 25M parameters
|
50 |
- Based on Phi-3 architecture
|