Initial commit

README.md

@@ -0,0 +1,101 @@
+---
+language: ca
+tags:
+- "catalan"
+metrics:
+- accuracy
+widget:
+ - text: "Ets més petita que un barrufet!!"
+ - text: "Ets tan lletja que et donaven de menjar per sota la porta."
+ 
+---
+# roberta-base-ca-finetuned-cyberbullying-catalan
+
+This model is a fine-tuned version of [BSC-TeMU/roberta-base-ca](https://huggingface.co/BSC-TeMU/roberta-base-ca) on the dataset generated scrapping all social networks (Twitter, Youtube ...) to detect cyberbullying on Catalan.
+
+It achieves the following results on the evaluation set:
+- Loss: 0.1508
+- Accuracy: 0.9665
+
+## Training and evaluation data
+
+I use the concatenation from multiple datasets generated scrapping social networks (Twitter,Youtube,Discord...)  to fine-tune this model. The total number of sentence pairs is above 410k sentences. Trained similar method at [roberta-base-bne-finetuned-cyberbullying-spanish](https://huggingface.co/JonatanGk/roberta-base-bne-finetuned-cyberbullying-spanish)
+
+## Training procedure
+
+<details>
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 4
+
+</details>
+
+### Model in action 🚀
+
+Fast usage with **pipelines**:
+
+```python
+from transformers import pipeline
+
+model_path = "JonatanGk/roberta-base-ca-finetuned-ciberbullying-catalan"
+bullying_analysis = pipeline("text-classification", model=model_path, tokenizer=model_path)
+
+bullying_analysis(
+    "Des que et vaig veure m'en vaig enamorar de tu."
+    )
+    
+# Output:
+[{'label': 'Not_bullying', 'score': 0.9996786117553711}]
+
+bullying_analysis(
+    "Ets tan lletja que et donaven de menjar per sota la porta."
+    )
+    
+# Output:
+[{'label': 'Bullying', 'score': 0.9927878975868225}]
+    
+```
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JonatanGk/Shared-Colab/blob/main/Cyberbullying_detection_(CATALAN).ipynb)
+
+### Framework versions
+- Transformers 4.10.3
+- Pytorch 1.9.0+cu102
+- Datasets 1.12.1
+- Tokenizers 0.10.3
+
+## Citation
+
+```bibtex
+@inproceedings{armengol-estape-etal-2021-multilingual,
+    title = "Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? {A} Comprehensive Assessment for {C}atalan",
+    author = "Armengol-Estap{\'e}, Jordi  and
+      Carrino, Casimiro Pio  and
+      Rodriguez-Penagos, Carlos  and
+      de Gibert Bonet, Ona  and
+      Armentano-Oller, Carme  and
+      Gonzalez-Agirre, Aitor  and
+      Melero, Maite  and
+      Villegas, Marta",
+    booktitle = "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021",
+    month = aug,
+    year = "2021",
+    address = "Online",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2021.findings-acl.437",
+    doi = "10.18653/v1/2021.findings-acl.437",
+    pages = "4933--4946",
+}
+```
+
+> Special thx to [Manuel Romero/@mrm8488](https://huggingface.co/mrm8488) as my mentor & R.C.
+
+> Created by [Jonatan Luna](https://JonatanGk.github.io) | [LinkedIn](https://www.linkedin.com/in/JonatanGk/)

config.json

@@ -0,0 +1,37 @@
+{
+  "_name_or_path": "BSC-TeMU/roberta-base-ca",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "id2label": {
+    "0": "Not_bullying",
+    "1": "Bullying"
+  },
+  "label2id": {
+    "Not_bullying": 0,
+    "Bullying": 1
+  },
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.11.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 52000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:782c9c811f47c511fc6c34a5ef24a92cbf1403883724905637026f9420ad0ee6
+size 504004013

special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "unk_token": {"content": "<unk>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "sep_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "cls_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true}}

tokenizer_config.json

@@ -0,0 +1 @@
+{"unk_token": {"content": "<unk>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "bos_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": true, "errors": "replace", "sep_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "cls_token": {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "max_len": 512, "special_tokens_map_file": null, "name_or_path": "BSC-TeMU/roberta-base-ca", "tokenizer_class": "RobertaTokenizer"}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61c9b71fc5afb32500a2283a9acb9a218b484c80a0e5e25484e6d5c91d2aceba
+size 2991