Commit
•
77ec9e2
1
Parent(s):
5fdd949
Update README.md
Browse files
README.md
CHANGED
@@ -18,9 +18,31 @@ widget:
|
|
18 |
---
|
19 |
# TeenyTinyLlama-162m-IMDB
|
20 |
|
21 |
-
TeenyTinyLlama is a series of small foundational models trained in Portuguese.
|
22 |
|
23 |
-
This repository contains a version of [TeenyTinyLlama-162m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-162m) fine-tuned on
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
|
25 |
## Reproducing
|
26 |
|
@@ -96,10 +118,34 @@ trainer = Trainer(
|
|
96 |
trainer.train()
|
97 |
```
|
98 |
|
99 |
-
##
|
100 |
|
101 |
| Models | [IMDB](https://huggingface.co/datasets/christykoh/imdb_pt) |
|
102 |
|--------------------------------------------------------------------------------------------|------------------------------------------------------------|
|
103 |
| [Teeny Tiny Llama 162m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-162m) | 91.14 |
|
104 |
| [Bert-base-portuguese-cased](https://huggingface.co/neuralmind/bert-base-portuguese-cased) | 92.22 |
|
105 |
| [Gpt2-small-portuguese](https://huggingface.co/pierreguillou/gpt2-small-portuguese) | 91.60 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
---
|
19 |
# TeenyTinyLlama-162m-IMDB
|
20 |
|
21 |
+
TeenyTinyLlama is a series of small foundational models trained in Brazilian Portuguese.
|
22 |
|
23 |
+
This repository contains a version of [TeenyTinyLlama-162m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-162m) (`TeenyTinyLlama-162m-IMDB`) fine-tuned on the the [IMDB dataset](https://huggingface.co/datasets/christykoh/imdb_pt).
|
24 |
+
|
25 |
+
## Details
|
26 |
+
|
27 |
+
- **Number of Epochs:** 3
|
28 |
+
- **Batch size:** 16
|
29 |
+
- **Optimizer:** `torch.optim.AdamW` (learning_rate = 4e-5, epsilon = 1e-8)
|
30 |
+
- **GPU:** 1 NVIDIA A100-SXM4-40GB
|
31 |
+
|
32 |
+
## Usage
|
33 |
+
|
34 |
+
Using `transformers.pipeline`:
|
35 |
+
|
36 |
+
```python
|
37 |
+
from transformers import pipeline
|
38 |
+
|
39 |
+
text = "Esqueceram de mim 2 é um dos melhores filmes de natal de todos os tempos."
|
40 |
+
|
41 |
+
classifier = pipeline("text-classification", model="nicholasKluge/TeenyTinyLlama-162m-IMDB")
|
42 |
+
classifier(text)
|
43 |
+
|
44 |
+
# >>> [{'label': 'POSITIVE', 'score': 0.9971244931221008}]
|
45 |
+
```
|
46 |
|
47 |
## Reproducing
|
48 |
|
|
|
118 |
trainer.train()
|
119 |
```
|
120 |
|
121 |
+
## Fine-Tuning Comparisons
|
122 |
|
123 |
| Models | [IMDB](https://huggingface.co/datasets/christykoh/imdb_pt) |
|
124 |
|--------------------------------------------------------------------------------------------|------------------------------------------------------------|
|
125 |
| [Teeny Tiny Llama 162m](https://huggingface.co/nicholasKluge/TeenyTinyLlama-162m) | 91.14 |
|
126 |
| [Bert-base-portuguese-cased](https://huggingface.co/neuralmind/bert-base-portuguese-cased) | 92.22 |
|
127 |
| [Gpt2-small-portuguese](https://huggingface.co/pierreguillou/gpt2-small-portuguese) | 91.60 |
|
128 |
+
|
129 |
+
## Cite as 🤗
|
130 |
+
|
131 |
+
```latex
|
132 |
+
|
133 |
+
@misc{nicholas22llama,
|
134 |
+
doi = {10.5281/zenodo.6989727},
|
135 |
+
url = {https://huggingface.co/nicholasKluge/TeenyTinyLlama-162m},
|
136 |
+
author = {Nicholas Kluge Corrêa},
|
137 |
+
title = {TeenyTinyLlama},
|
138 |
+
year = {2023},
|
139 |
+
publisher = {HuggingFace},
|
140 |
+
journal = {HuggingFace repository},
|
141 |
+
}
|
142 |
+
|
143 |
+
```
|
144 |
+
|
145 |
+
## Funding
|
146 |
+
|
147 |
+
This repository was built as part of the RAIES ([Rede de Inteligência Artificial Ética e Segura](https://www.raies.org/)) initiative, a project supported by FAPERGS - ([Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul](https://fapergs.rs.gov.br/inicial)), Brazil.
|
148 |
+
|
149 |
+
## License
|
150 |
+
|
151 |
+
The TeenyTinyLlama-162m-IMDB is licensed under the Apache License, Version 2.0. See the [LICENSE](LICENSE) file for more details.
|