pasithbas159/multilabel-cls-model-v3

README.md

@@ -0,0 +1,92 @@
+---
+library_name: transformers
+license: mit
+base_model: xlm-roberta-large
+tags:
+- generated_from_trainer
+metrics:
+- f1
+- accuracy
+model-index:
+- name: multilabel_transfer_learning_transformer
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# multilabel_transfer_learning_transformer
+
+This model is a fine-tuned version of [xlm-roberta-large](https://huggingface.co/xlm-roberta-large) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0217
+- F1: 0.9924
+- Roc Auc: 0.9955
+- Accuracy: 0.9887
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1e-05
+- train_batch_size: 8
+- eval_batch_size: 4
+- seed: 123
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_steps: 300
+- num_epochs: 100
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | F1     | Roc Auc | Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:------:|:-------:|:--------:|
+| 0.5454        | 1.0   | 136  | 0.4135          | 0.0125 | 0.5030  | 0.0      |
+| 0.3917        | 2.0   | 272  | 0.3582          | 0.2939 | 0.5855  | 0.0338   |
+| 0.3405        | 3.0   | 408  | 0.3048          | 0.4862 | 0.6649  | 0.0827   |
+| 0.2918        | 4.0   | 544  | 0.2753          | 0.5913 | 0.7250  | 0.1278   |
+| 0.2531        | 5.0   | 680  | 0.2285          | 0.7261 | 0.8065  | 0.2406   |
+| 0.214         | 6.0   | 816  | 0.1971          | 0.7684 | 0.8328  | 0.3233   |
+| 0.181         | 7.0   | 952  | 0.1663          | 0.8199 | 0.8624  | 0.4173   |
+| 0.1529        | 8.0   | 1088 | 0.1431          | 0.8591 | 0.8905  | 0.4774   |
+| 0.1307        | 9.0   | 1224 | 0.1224          | 0.8979 | 0.9260  | 0.6090   |
+| 0.1108        | 10.0  | 1360 | 0.1034          | 0.9195 | 0.9329  | 0.6955   |
+| 0.0961        | 11.0  | 1496 | 0.0920          | 0.9435 | 0.9553  | 0.7744   |
+| 0.0821        | 12.0  | 1632 | 0.0793          | 0.9559 | 0.9627  | 0.8346   |
+| 0.0719        | 13.0  | 1768 | 0.0682          | 0.9636 | 0.9732  | 0.8759   |
+| 0.0612        | 14.0  | 1904 | 0.0618          | 0.9651 | 0.9760  | 0.8947   |
+| 0.0526        | 15.0  | 2040 | 0.0519          | 0.9757 | 0.9796  | 0.9135   |
+| 0.0456        | 16.0  | 2176 | 0.0468          | 0.9778 | 0.9835  | 0.9248   |
+| 0.0394        | 17.0  | 2312 | 0.0396          | 0.9854 | 0.9885  | 0.9586   |
+| 0.0343        | 18.0  | 2448 | 0.0372          | 0.9855 | 0.9911  | 0.9586   |
+| 0.0299        | 19.0  | 2584 | 0.0329          | 0.9854 | 0.9885  | 0.9586   |
+| 0.0266        | 20.0  | 2720 | 0.0289          | 0.9887 | 0.9932  | 0.9887   |
+| 0.0233        | 21.0  | 2856 | 0.0264          | 0.9874 | 0.9919  | 0.9812   |
+| 0.0212        | 22.0  | 2992 | 0.0258          | 0.9887 | 0.9932  | 0.9887   |
+| 0.02          | 23.0  | 3128 | 0.0242          | 0.9887 | 0.9932  | 0.9887   |
+| 0.0177        | 24.0  | 3264 | 0.0217          | 0.9924 | 0.9955  | 0.9887   |
+| 0.0162        | 25.0  | 3400 | 0.0200          | 0.9887 | 0.9932  | 0.9887   |
+| 0.0146        | 26.0  | 3536 | 0.0201          | 0.9906 | 0.9951  | 0.9887   |
+| 0.0136        | 27.0  | 3672 | 0.0192          | 0.9906 | 0.9951  | 0.9887   |
+| 0.0127        | 28.0  | 3808 | 0.0169          | 0.9924 | 0.9955  | 0.9887   |
+
+
+### Framework versions
+
+- Transformers 4.44.2
+- Pytorch 2.4.1+cu121
+- Datasets 3.0.1
+- Tokenizers 0.19.1

config.json

@@ -0,0 +1,69 @@
+{
+  "_name_or_path": "xlm-roberta-large",
+  "architectures": [
+    "XLMRobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.15,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.15,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "CE",
+    "1": "ENV",
+    "2": "BME",
+    "3": "PE",
+    "4": "METAL",
+    "5": "ME",
+    "6": "EE",
+    "7": "CPE",
+    "8": "OPTIC",
+    "9": "NANO",
+    "10": "CHE",
+    "11": "MATENG",
+    "12": "AGRI",
+    "13": "EDU",
+    "14": "IE",
+    "15": "SAFETY",
+    "16": "MATH",
+    "17": "MATSCI"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "AGRI": 12,
+    "BME": 2,
+    "CE": 0,
+    "CHE": 10,
+    "CPE": 7,
+    "EDU": 13,
+    "EE": 6,
+    "ENV": 1,
+    "IE": 14,
+    "MATENG": 11,
+    "MATH": 16,
+    "MATSCI": 17,
+    "ME": 5,
+    "METAL": 4,
+    "NANO": 9,
+    "OPTIC": 8,
+    "PE": 3,
+    "SAFETY": 15
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "xlm-roberta",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "output_past": true,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "multi_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 250002
+}

logs/events.out.tfevents.1728650631.2082b953c793.5832.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:972b69bb02398c2e400549cf166b524efcf8df469d801426090342c58866aa00
+size 23684

logs/events.out.tfevents.1728652713.2082b953c793.5832.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdce3b11c741f057733a1b4a9e76fbaaaa95ae8c08a2f0b4e3abb58cf23e77d2
+size 508

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3185d20b472f9f5efafd4363ef62a821e50d6bd292ec4af5614014a10be4a551
+size 2239684272

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64accda8622a690125e7212e65b6344f3eef1685badbadcde8218b613fe5bfe2
+size 5176