nicholasKluge commited on
Commit
1ae92de
1 Parent(s): e250347

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -3
README.md CHANGED
@@ -186,7 +186,7 @@ model-index:
186
 
187
  Large language models (LLMs) have significantly advanced natural language processing, but their progress has yet to be equal across languages. While most LLMs are trained in high-resource languages like English, multilingual models generally underperform monolingual ones. Additionally, aspects of their multilingual foundation sometimes restrict the byproducts they produce, like computational demands and licensing regimes. Hence, we developed the _TeenyTinyLlama_ pair: two compact models for Brazilian Portuguese text generation.
188
 
189
- Read our preprint on [ArXiv](https://arxiv.org/abs/2401.16640).
190
 
191
  ## Details
192
 
@@ -195,7 +195,7 @@ Read our preprint on [ArXiv](https://arxiv.org/abs/2401.16640).
195
  - **Context length:** 2048 tokens
196
  - **Dataset:** [Pt-Corpus Instruct](https://huggingface.co/datasets/nicholasKluge/Pt-Corpus-Instruct) (6.2B tokens)
197
  - **Language:** Portuguese
198
- - **Number of steps:** 458,000
199
  - **GPU:** 1 NVIDIA A100-SXM4-40GB
200
  - **Training time**: ~ 36 hours
201
  - **Emissions:** 5.6 KgCO2 (Germany)
@@ -355,7 +355,6 @@ All the shown results are the higher accuracy scores achieved on the respective
355
  ## Cite as 🤗
356
 
357
  ```latex
358
-
359
  @misc{correa24ttllama,
360
  title = {TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguese},
361
  author = {Corr{\^e}a, Nicholas Kluge and Falk, Sophia and Fatimah, Shiza and Sen, Aniket and De Oliveira, Nythamar},
@@ -363,6 +362,15 @@ All the shown results are the higher accuracy scores achieved on the respective
363
  year={2024}
364
  }
365
 
 
 
 
 
 
 
 
 
 
366
  ```
367
 
368
  ## Funding
 
186
 
187
  Large language models (LLMs) have significantly advanced natural language processing, but their progress has yet to be equal across languages. While most LLMs are trained in high-resource languages like English, multilingual models generally underperform monolingual ones. Additionally, aspects of their multilingual foundation sometimes restrict the byproducts they produce, like computational demands and licensing regimes. Hence, we developed the _TeenyTinyLlama_ pair: two compact models for Brazilian Portuguese text generation.
188
 
189
+ Read our preprint on [Article](https://www.sciencedirect.com/science/article/pii/S2666827024000343).
190
 
191
  ## Details
192
 
 
195
  - **Context length:** 2048 tokens
196
  - **Dataset:** [Pt-Corpus Instruct](https://huggingface.co/datasets/nicholasKluge/Pt-Corpus-Instruct) (6.2B tokens)
197
  - **Language:** Portuguese
198
+ - **Number of steps:** 458,000
199
  - **GPU:** 1 NVIDIA A100-SXM4-40GB
200
  - **Training time**: ~ 36 hours
201
  - **Emissions:** 5.6 KgCO2 (Germany)
 
355
  ## Cite as 🤗
356
 
357
  ```latex
 
358
  @misc{correa24ttllama,
359
  title = {TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguese},
360
  author = {Corr{\^e}a, Nicholas Kluge and Falk, Sophia and Fatimah, Shiza and Sen, Aniket and De Oliveira, Nythamar},
 
362
  year={2024}
363
  }
364
 
365
+ @misc{correa24ttllama,
366
+ doi = {10.1016/j.mlwa.2024.100558},
367
+ url = {https://www.sciencedirect.com/science/article/pii/S2666827024000343},
368
+ title = {TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguese},
369
+ author = {Corr{\^e}a, Nicholas Kluge and Falk, Sophia and Fatimah, Shiza and Sen, Aniket and De Oliveira, Nythamar},
370
+ journal={Machine Learning With Applications},
371
+ publisher = {Springer},
372
+ year={2024}
373
+ }
374
  ```
375
 
376
  ## Funding