Commit
•
1ee6bdc
1
Parent(s):
e41caee
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
language:
|
3 |
+
- tl
|
4 |
+
tags:
|
5 |
+
- RoBERTa
|
6 |
+
- Tagalog
|
7 |
+
- Taglish
|
8 |
+
---
|
9 |
+
|
10 |
+
## Model Description
|
11 |
+
|
12 |
+
|
13 |
+
As part of the ITANONG project's 10 billion-token Tagalog dataset, we have introduced our initial pre-trained language models for Philippine languages. Our model suite encompasses various BERT-based, GPT-based, and Sentence Transformers tailored for Tagalog,Taglish and Cebuano.
|
14 |
+
This RoBERTa model was used in the original paper for sentiment analysis encompassing three tags (positive, negative, neutral) for code-switched Tagalog.
|
15 |
+
|
16 |
+
|
17 |
+
|
18 |
+
## Training Details
|
19 |
+
This model was trained using an Nvidia V100-32GB GPU on DOST-ASTI Computing and Archiving Research Environment (COARE) - https://asti.dost.gov.ph/projects/coare/
|
20 |
+
|
21 |
+
### Training Data
|
22 |
+
The training dataset was compiled from both formal and informal sources, consisting of 5,159,917 instances from formal channels and 3,057,180 from informal sources. More information on pre-processing and training parameters on our paper
|
23 |
+
|
24 |
+
|
25 |
+
## Citation
|
26 |
+
Paper : iTANONG-DS : A Collection of Benchmark Datasets for Downstream Natural Language Processing Tasks on Select Philippine Language
|
27 |
+
|
28 |
+
|
29 |
+
Bibtex:
|
30 |
+
```
|
31 |
+
@inproceedings{visperas-etal-2023-itanong,
|
32 |
+
title = "i{TANONG}-{DS} : A Collection of Benchmark Datasets for Downstream Natural Language Processing Tasks on Select {P}hilippine Languages",
|
33 |
+
author = "Visperas, Moses L. and
|
34 |
+
Borjal, Christalline Joie and
|
35 |
+
Adoptante, Aunhel John M and
|
36 |
+
Abacial, Danielle Shine R. and
|
37 |
+
Decano, Ma. Miciella and
|
38 |
+
Peramo, Elmer C",
|
39 |
+
editor = "Abbas, Mourad and
|
40 |
+
Freihat, Abed Alhakim",
|
41 |
+
booktitle = "Proceedings of the 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023)",
|
42 |
+
month = dec,
|
43 |
+
year = "2023",
|
44 |
+
address = "Online",
|
45 |
+
publisher = "Association for Computational Linguistics",
|
46 |
+
url = "https://aclanthology.org/2023.icnlsp-1.34",
|
47 |
+
pages = "316--323",
|
48 |
+
}
|
49 |
+
```
|
50 |
+
|
51 |
+
|