Update README.md

README.md

@@ -1,58 +1,190 @@
 ---
-base_model: xlm-r-large-script_expand
+license: mit
 tags:
 - generated_from_trainer
-metrics:
-- accuracy
 model-index:
-- name: afro-xlmr_large_76L_script
+- name: afro-xlmr-large-76L_script
   results: []
+language:
+- en
+- am
+- ar
+- so
+- sw
+- pt
+- af
+- fr
+- zu
+- mg
+- ha
+- sn
+- arz
+- ny
+- ig
+- xh
+- yo
+- st
+- rw
+- tn
+- ti
+- ts
+- om
+- run
+- nso
+- ee
+- ln
+- tw
+- pcm
+- gaa
+- loz
+- lg
+- guw
+- bem
+- efi
+- lue
+- lua
+- toi
+- ve
+- tum
+- tll
+- iso
+- kqn
+- zne
+- umb
+- mos
+- tiv
+- lu
+- ff
+- kwy
+- bci
+- rnd
+- luo
+- wal
+- ss
+- lun
+- wo
+- nyk
+- kj
+- ki
+- fon
+- bm
+- cjk
+- din
+- dyu
+- kab
+- kam
+- kbp
+- kr
+- kmb
+- kg
+- nus
+- sg
+- taq
+- tzm
+- nqo
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
 
-# afro-xlmr_large_76L_script
-
-This model is a fine-tuned version of [xlm-r-large-script_expand](https://huggingface.co/xlm-r-large-script_expand) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 1.0044
-- Accuracy: 0.7963
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 40
-- eval_batch_size: 32
-- seed: 42
-- gradient_accumulation_steps: 8
-- total_train_batch_size: 320
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- num_epochs: 3.0
-
-### Training results
-
-
-
-### Framework versions
-
-- Transformers 4.34.1
-- Pytorch 2.1.0+cu121
-- Datasets 2.14.5
-- Tokenizers 0.14.1
+# afro-xlmr-large-75L_script
+
+AfroXLMR-large was created by first augmenting the XLM-R-large model with missing scripts (N'Ko and Tifinagh), followed by an MLM adaptation of the expanded XLM-R-large model on 76 languages widely spoken in Africa
+including 4 high-resource languages. 
+
+### Pre-training corpus
+A mix of mC4, Wikipedia and OPUS data
+
+### Languages
+
+There are 75 languages available :
+- English (eng)
+- Amharic (amh)
+- Arabic (ara) 
+- Somali (som)
+- Kiswahili (swa)
+- Portuguese (por)
+- Afrikaans (afr)
+- French (fra)
+- isiZulu (zul)
+- Malagasy (mlg)
+- Hausa (hau)
+- chiShona (sna)
+- Egyptian Arabic (arz)
+- Chichewa (nya)
+- Igbo (ibo)
+- isiXhosa (xho)
+- Yorùbá (yor)
+- Sesotho (sot)
+- Kinyarwanda (kin)
+- Tigrinya (tir)
+- Tsonga (tso)
+- Oromo (orm)
+- Rundi (run)
+- Northern Sotho (nso)
+- Ewe (ewe)
+- Lingala (lin)
+- Twi (twi)
+- Nigerian Pidgin (pcm)
+- Ga (gaa)
+- Lozi (loz)
+- Luganda (lug)
+- Gun (guw)
+- Bemba (bem)
+- Efik (efi)
+- Luvale (lue) 
+- Luba-Lulua (lua)
+- Tonga (toi)
+- Tshivenḓa (ven)
+- Tumbuka (tum)
+- Tetela (tll)
+- Isoko (iso)
+- Kaonde (kqn)
+- Zande (zne)
+- Umbundu (umb)
+- Mossi (mos)
+- Tiv (tiv)
+- Luba-Katanga (lub)
+- Fula (fuv)
+- San Salvador Kongo (kwy)
+- Baoulé (bci)
+- Ruund (rnd)
+- Luo (luo)
+- Wolaitta (wal) 
+- Swazi (ssw)
+- Lunda (lun)
+- Wolof (wol)
+- Nyaneka (nyk) 
+- Kwanyama (kua)
+- Kikuyu (kik)
+- Fon (fon)
+- Bambara (bam)
+- Chokwe (cjk)
+- Dinka (dik)
+- Dyula (dyu)
+- Kabyle (kab)
+- Kamba (kam)
+- Kabiyè (kbp)
+- Kanuri (knc)
+- Kimbundu (kmb)
+- Kikongo (kon)
+- Nuer (nus)
+- Sango (sag)
+- Tamasheq (taq)
+- Tamazight (tzm)
+
+
+### Acknowledgment
+We would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch.
+
+
+### BibTeX entry and citation info.
+```
+@misc{adelani2023sib200,
+      title={SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects}, 
+      author={David Ifeoluwa Adelani and Hannah Liu and Xiaoyu Shen and Nikita Vassilyev and Jesujoba O. Alabi and Yanke Mao and Haonan Gao and Annie En-Shiun Lee},
+      year={2023},
+      eprint={2309.07445},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+
+```