afrizalha commited on
Commit
0607ca5
•
1 Parent(s): 9c6f68b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -37,6 +37,14 @@ inference: true
37
  ### 🎻 Welcome!
38
  Sasando-1 is a tiny, highly experimental Indonesian text generator built using the Phi-3 architecture. It comes with two variations of microscopic sizes: 7M and 25M parameters. It is trained on a tightly-controlled Indo4B dataset filtered to only have 18000 unique words. The method is inspired by Microsoft's TinyStories paper which demonstrates that a tiny language model can produce fluent text when trained on tightly-controlled dataset.
39
 
 
 
 
 
 
 
 
 
40
  ### ✨ Specs
41
  - Comes with 7M and 25M parameters
42
  - Based on Phi-3 architecture
 
37
  ### 🎻 Welcome!
38
  Sasando-1 is a tiny, highly experimental Indonesian text generator built using the Phi-3 architecture. It comes with two variations of microscopic sizes: 7M and 25M parameters. It is trained on a tightly-controlled Indo4B dataset filtered to only have 18000 unique words. The method is inspired by Microsoft's TinyStories paper which demonstrates that a tiny language model can produce fluent text when trained on tightly-controlled dataset.
39
 
40
+ ### 🇮🇩 Context
41
+ Indonesia has +700 languages, and many of them are dying at an alarming rate. Language technologies like generative AI can play a massive role in language preservation. However, Indonesia has several contextual issues:
42
+
43
+ - Many languages, including those with millions of speakers, have low-volume digital resources
44
+ - Running large models can be costly, while Indonesia is a middle-income country with little funding
45
+
46
+ Overcoming these challenges require developers to work with what little data and money that they have. Sasando-1 is a prototypical demonstration that thinly-available resources can potentially still be leveraged to develop generative models with cheap compute.
47
+
48
  ### ✨ Specs
49
  - Comes with 7M and 25M parameters
50
  - Based on Phi-3 architecture