Hugging Face's logo

haryoaw
/
scenario-KD-PO-CDF-ALL-D2_data-cardiffnlp_tweet_sentiment_multilingual_all66sss

Transformers
Safetensors
deberta-v2
Generated from Trainer
Inference Endpoints
Model card Files Community
haryoaw commited on
Commit
b860755
•
1 Parent(s): 4701845

Initial Commit

Browse files
Files changed (5) hide show
  1. README.md +111 -0
  2. config.json +45 -0
  3. eval_results_cardiff.json +1 -0
  4. model.safetensors +3 -0
  5. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: mit
4
+ base_model: haryoaw/scenario-MDBT-TCR_data-cardiffnlp_tweet_sentiment_multilingual_all
5
+ tags:
6
+ - generated_from_trainer
7
+ datasets:
8
+ - tweet_sentiment_multilingual
9
+ metrics:
10
+ - accuracy
11
+ - f1
12
+ model-index:
13
+ - name: scenario-KD-PO-CDF-ALL-D2_data-cardiffnlp_tweet_sentiment_multilingual_all66sss
14
+ results: []
15
+ ---
16
+
17
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
18
+ should probably proofread and complete it, then remove this comment. -->
19
+
20
+ # scenario-KD-PO-CDF-ALL-D2_data-cardiffnlp_tweet_sentiment_multilingual_all66sss
21
+
22
+ This model is a fine-tuned version of [haryoaw/scenario-MDBT-TCR_data-cardiffnlp_tweet_sentiment_multilingual_all](https://huggingface.co/haryoaw/scenario-MDBT-TCR_data-cardiffnlp_tweet_sentiment_multilingual_all) on the tweet_sentiment_multilingual dataset.
23
+ It achieves the following results on the evaluation set:
24
+ - Loss: 125.3087
25
+ - Accuracy: 0.5552
26
+ - F1: 0.5557
27
+
28
+ ## Model description
29
+
30
+ More information needed
31
+
32
+ ## Intended uses & limitations
33
+
34
+ More information needed
35
+
36
+ ## Training and evaluation data
37
+
38
+ More information needed
39
+
40
+ ## Training procedure
41
+
42
+ ### Training hyperparameters
43
+
44
+ The following hyperparameters were used during training:
45
+ - learning_rate: 5e-05
46
+ - train_batch_size: 8
47
+ - eval_batch_size: 32
48
+ - seed: 66
49
+ - gradient_accumulation_steps: 4
50
+ - total_train_batch_size: 32
51
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
52
+ - lr_scheduler_type: linear
53
+ - num_epochs: 50
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
58
+ |:-------------:|:-------:|:-----:|:---------------:|:--------:|:------:|
59
+ | 196.6841 | 1.0875 | 500 | 166.6670 | 0.3484 | 0.2582 |
60
+ | 168.7515 | 2.1751 | 1000 | 157.8051 | 0.4136 | 0.4041 |
61
+ | 160.6266 | 3.2626 | 1500 | 153.1391 | 0.4471 | 0.4197 |
62
+ | 152.2116 | 4.3502 | 2000 | 150.9196 | 0.4225 | 0.4031 |
63
+ | 145.1439 | 5.4377 | 2500 | 145.2500 | 0.4988 | 0.4938 |
64
+ | 137.2401 | 6.5253 | 3000 | 145.8920 | 0.4992 | 0.4899 |
65
+ | 129.4263 | 7.6128 | 3500 | 139.7630 | 0.5123 | 0.5121 |
66
+ | 122.0069 | 8.7004 | 4000 | 142.0804 | 0.5096 | 0.4979 |
67
+ | 116.0619 | 9.7879 | 4500 | 142.2508 | 0.5297 | 0.5245 |
68
+ | 110.0063 | 10.8755 | 5000 | 136.2582 | 0.5444 | 0.5442 |
69
+ | 105.7 | 11.9630 | 5500 | 133.7262 | 0.5390 | 0.5382 |
70
+ | 101.5969 | 13.0506 | 6000 | 135.1363 | 0.5370 | 0.5366 |
71
+ | 98.3408 | 14.1381 | 6500 | 135.0693 | 0.5417 | 0.5426 |
72
+ | 95.2212 | 15.2257 | 7000 | 134.8993 | 0.5494 | 0.5498 |
73
+ | 92.8186 | 16.3132 | 7500 | 132.5809 | 0.5328 | 0.5321 |
74
+ | 90.2266 | 17.4008 | 8000 | 131.8017 | 0.5328 | 0.5343 |
75
+ | 88.4703 | 18.4883 | 8500 | 131.0873 | 0.5235 | 0.5213 |
76
+ | 87.1873 | 19.5759 | 9000 | 131.4287 | 0.5382 | 0.5379 |
77
+ | 85.7516 | 20.6634 | 9500 | 131.1887 | 0.5455 | 0.5448 |
78
+ | 84.7201 | 21.7510 | 10000 | 129.0503 | 0.5463 | 0.5463 |
79
+ | 83.2881 | 22.8385 | 10500 | 130.1761 | 0.5432 | 0.5438 |
80
+ | 82.0884 | 23.9260 | 11000 | 128.6964 | 0.5355 | 0.5366 |
81
+ | 81.8031 | 25.0136 | 11500 | 128.3357 | 0.5421 | 0.5415 |
82
+ | 80.7617 | 26.1011 | 12000 | 129.1492 | 0.5428 | 0.5441 |
83
+ | 80.189 | 27.1887 | 12500 | 127.5801 | 0.5552 | 0.5565 |
84
+ | 79.3389 | 28.2762 | 13000 | 127.4090 | 0.5370 | 0.5387 |
85
+ | 78.7626 | 29.3638 | 13500 | 129.6783 | 0.5505 | 0.5514 |
86
+ | 78.3347 | 30.4513 | 14000 | 129.3366 | 0.5517 | 0.5505 |
87
+ | 77.8829 | 31.5389 | 14500 | 128.1779 | 0.5421 | 0.5412 |
88
+ | 77.4036 | 32.6264 | 15000 | 128.5850 | 0.5309 | 0.5308 |
89
+ | 76.9029 | 33.7140 | 15500 | 126.4255 | 0.5509 | 0.5509 |
90
+ | 76.5128 | 34.8015 | 16000 | 126.3467 | 0.5424 | 0.5429 |
91
+ | 76.2921 | 35.8891 | 16500 | 125.5003 | 0.5428 | 0.5446 |
92
+ | 75.861 | 36.9766 | 17000 | 126.6583 | 0.5486 | 0.5496 |
93
+ | 75.5675 | 38.0642 | 17500 | 126.0611 | 0.5525 | 0.5519 |
94
+ | 75.2374 | 39.1517 | 18000 | 126.2538 | 0.5451 | 0.5457 |
95
+ | 75.1584 | 40.2393 | 18500 | 126.7418 | 0.5463 | 0.5448 |
96
+ | 74.9139 | 41.3268 | 19000 | 126.7482 | 0.5505 | 0.5491 |
97
+ | 74.565 | 42.4144 | 19500 | 126.0126 | 0.5494 | 0.5507 |
98
+ | 74.3026 | 43.5019 | 20000 | 124.4608 | 0.5563 | 0.5573 |
99
+ | 74.4625 | 44.5895 | 20500 | 125.6297 | 0.5563 | 0.5569 |
100
+ | 74.1991 | 45.6770 | 21000 | 125.5350 | 0.5498 | 0.5497 |
101
+ | 74.0552 | 46.7645 | 21500 | 124.8100 | 0.5552 | 0.5560 |
102
+ | 74.0304 | 47.8521 | 22000 | 126.1095 | 0.5482 | 0.5483 |
103
+ | 73.8794 | 48.9396 | 22500 | 125.3087 | 0.5552 | 0.5557 |
104
+
105
+
106
+ ### Framework versions
107
+
108
+ - Transformers 4.44.2
109
+ - Pytorch 2.1.1+cu121
110
+ - Datasets 2.14.5
111
+ - Tokenizers 0.19.1
config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "haryoaw/scenario-MDBT-TCR_data-cardiffnlp_tweet_sentiment_multilingual_all",
3
+ "architectures": [
4
+ "DebertaForSequenceClassificationKD"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "hidden_act": "gelu",
8
+ "hidden_dropout_prob": 0.1,
9
+ "hidden_size": 384,
10
+ "id2label": {
11
+ "0": "LABEL_0",
12
+ "1": "LABEL_1",
13
+ "2": "LABEL_2"
14
+ },
15
+ "initializer_range": 0.02,
16
+ "intermediate_size": 1536,
17
+ "label2id": {
18
+ "LABEL_0": 0,
19
+ "LABEL_1": 1,
20
+ "LABEL_2": 2
21
+ },
22
+ "layer_norm_eps": 1e-07,
23
+ "max_position_embeddings": 512,
24
+ "max_relative_positions": -1,
25
+ "model_type": "deberta-v2",
26
+ "norm_rel_ebd": "layer_norm",
27
+ "num_attention_heads": 12,
28
+ "num_hidden_layers": 6,
29
+ "pad_token_id": 0,
30
+ "pooler_dropout": 0,
31
+ "pooler_hidden_act": "gelu",
32
+ "pooler_hidden_size": 384,
33
+ "pos_att_type": [
34
+ "p2c",
35
+ "c2p"
36
+ ],
37
+ "position_biased_input": false,
38
+ "position_buckets": 256,
39
+ "relative_attention": true,
40
+ "share_att_key": true,
41
+ "torch_dtype": "float32",
42
+ "transformers_version": "4.44.2",
43
+ "type_vocab_size": 0,
44
+ "vocab_size": 251000
45
+ }
eval_results_cardiff.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"arabic": {"f1": 0.47787031964301985, "accuracy": 0.47701149425287354, "confusion_matrix": [[155, 95, 40], [104, 128, 58], [82, 76, 132]]}, "english": {"f1": 0.6019344930728187, "accuracy": 0.6011494252873564, "confusion_matrix": [[198, 64, 28], [96, 156, 38], [45, 76, 169]]}, "french": {"f1": 0.5488778462116107, "accuracy": 0.5505747126436782, "confusion_matrix": [[172, 67, 51], [35, 182, 73], [60, 105, 125]]}, "german": {"f1": 0.6091779428804602, "accuracy": 0.6103448275862069, "confusion_matrix": [[152, 82, 56], [56, 185, 49], [38, 58, 194]]}, "hindi": {"f1": 0.46097082020728225, "accuracy": 0.4689655172413793, "confusion_matrix": [[191, 68, 31], [134, 119, 37], [118, 74, 98]]}, "italian": {"f1": 0.551721805199501, "accuracy": 0.5609195402298851, "confusion_matrix": [[107, 124, 59], [24, 212, 54], [38, 83, 169]]}, "portuguese": {"f1": 0.5829909471121991, "accuracy": 0.5839080459770115, "confusion_matrix": [[143, 101, 46], [65, 164, 61], [25, 64, 201]]}, "spanish": {"f1": 0.5465956212940437, "accuracy": 0.5436781609195402, "confusion_matrix": [[155, 105, 30], [69, 155, 66], [41, 86, 163]]}, "all": {"f1": 0.5519663461167889, "accuracy": 0.5508620689655173, "confusion_matrix": [[1265, 719, 336], [580, 1317, 423], [452, 616, 1252]]}}
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c0eb009fe0c66c940808b138a3d4bef1c5df87342aa6c980ebf64c906d396d5
3
+ size 430707332
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:045acf9944a39ae852580479050f5309b450ae10554559462f9b60d259aea6a0
3
+ size 5368