Hugging Face's logo

RonTon05
/
Roberta-CLS-URL

Text Classification
Transformers
Safetensors
xlm-roberta
Generated from Trainer
Inference Endpoints
Model card Files Community
RonTon05 commited on
Commit
7fbb3c9
·
verified ·
1 Parent(s): 3c10419

End of training

Browse files
Files changed (9) hide show
  1. .gitattributes +1 -0
  2. README.md +133 -0
  3. config.json +29 -0
  4. model.safetensors +3 -0
  5. sentencepiece.bpe.model +3 -0
  6. special_tokens_map.json +51 -0
  7. tokenizer.json +3 -0
  8. tokenizer_config.json +54 -0
  9. training_args.bin +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,133 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: papluca/xlm-roberta-base-language-detection
4
+ tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - accuracy
8
+ - f1
9
+ model-index:
10
+ - name: Roberta-CLS-URL
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # Roberta-CLS-URL
18
+
19
+ This model is a fine-tuned version of [papluca/xlm-roberta-base-language-detection](https://huggingface.co/papluca/xlm-roberta-base-language-detection) on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.1817
22
+ - Accuracy: 0.9571
23
+ - F1: 0.9572
24
+
25
+ ## Model description
26
+
27
+ More information needed
28
+
29
+ ## Intended uses & limitations
30
+
31
+ More information needed
32
+
33
+ ## Training and evaluation data
34
+
35
+ More information needed
36
+
37
+ ## Training procedure
38
+
39
+ ### Training hyperparameters
40
+
41
+ The following hyperparameters were used during training:
42
+ - learning_rate: 2e-05
43
+ - train_batch_size: 64
44
+ - eval_batch_size: 64
45
+ - seed: 42
46
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
47
+ - lr_scheduler_type: linear
48
+ - num_epochs: 10
49
+
50
+ ### Training results
51
+
52
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
53
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
54
+ | No log | 0.14 | 50 | 0.2787 | 0.8943 | 0.8942 |
55
+ | No log | 0.28 | 100 | 0.2332 | 0.9179 | 0.9180 |
56
+ | No log | 0.42 | 150 | 0.2369 | 0.9268 | 0.9269 |
57
+ | No log | 0.56 | 200 | 0.2071 | 0.9313 | 0.9314 |
58
+ | No log | 0.69 | 250 | 0.2017 | 0.9344 | 0.9343 |
59
+ | No log | 0.83 | 300 | 0.1953 | 0.9414 | 0.9415 |
60
+ | No log | 0.97 | 350 | 0.2031 | 0.9394 | 0.9394 |
61
+ | 0.2675 | 1.11 | 400 | 0.1915 | 0.9439 | 0.9439 |
62
+ | 0.2675 | 1.25 | 450 | 0.1696 | 0.9439 | 0.9440 |
63
+ | 0.2675 | 1.39 | 500 | 0.1747 | 0.9487 | 0.9488 |
64
+ | 0.2675 | 1.53 | 550 | 0.1958 | 0.9425 | 0.9427 |
65
+ | 0.2675 | 1.67 | 600 | 0.1608 | 0.9462 | 0.9461 |
66
+ | 0.2675 | 1.81 | 650 | 0.1547 | 0.9523 | 0.9524 |
67
+ | 0.2675 | 1.94 | 700 | 0.1668 | 0.9557 | 0.9557 |
68
+ | 0.1686 | 2.08 | 750 | 0.1709 | 0.9498 | 0.9499 |
69
+ | 0.1686 | 2.22 | 800 | 0.1605 | 0.9554 | 0.9555 |
70
+ | 0.1686 | 2.36 | 850 | 0.1703 | 0.9501 | 0.9501 |
71
+ | 0.1686 | 2.5 | 900 | 0.1603 | 0.9465 | 0.9466 |
72
+ | 0.1686 | 2.64 | 950 | 0.1742 | 0.9451 | 0.9451 |
73
+ | 0.1686 | 2.78 | 1000 | 0.1507 | 0.9546 | 0.9546 |
74
+ | 0.1686 | 2.92 | 1050 | 0.1423 | 0.9557 | 0.9557 |
75
+ | 0.1385 | 3.06 | 1100 | 0.1496 | 0.9574 | 0.9575 |
76
+ | 0.1385 | 3.19 | 1150 | 0.1590 | 0.9549 | 0.9549 |
77
+ | 0.1385 | 3.33 | 1200 | 0.1492 | 0.9523 | 0.9524 |
78
+ | 0.1385 | 3.47 | 1250 | 0.1390 | 0.9565 | 0.9566 |
79
+ | 0.1385 | 3.61 | 1300 | 0.1496 | 0.9529 | 0.9530 |
80
+ | 0.1385 | 3.75 | 1350 | 0.1425 | 0.9551 | 0.9552 |
81
+ | 0.1385 | 3.89 | 1400 | 0.1494 | 0.9521 | 0.9522 |
82
+ | 0.1221 | 4.03 | 1450 | 0.1541 | 0.9557 | 0.9557 |
83
+ | 0.1221 | 4.17 | 1500 | 0.1897 | 0.9532 | 0.9532 |
84
+ | 0.1221 | 4.31 | 1550 | 0.1595 | 0.9518 | 0.9519 |
85
+ | 0.1221 | 4.44 | 1600 | 0.1514 | 0.9554 | 0.9555 |
86
+ | 0.1221 | 4.58 | 1650 | 0.1553 | 0.9554 | 0.9555 |
87
+ | 0.1221 | 4.72 | 1700 | 0.1626 | 0.9543 | 0.9543 |
88
+ | 0.1221 | 4.86 | 1750 | 0.1509 | 0.9523 | 0.9523 |
89
+ | 0.1034 | 5.0 | 1800 | 0.1448 | 0.9554 | 0.9555 |
90
+ | 0.1034 | 5.14 | 1850 | 0.1685 | 0.9574 | 0.9574 |
91
+ | 0.1034 | 5.28 | 1900 | 0.1555 | 0.9551 | 0.9552 |
92
+ | 0.1034 | 5.42 | 1950 | 0.1595 | 0.9557 | 0.9557 |
93
+ | 0.1034 | 5.56 | 2000 | 0.1660 | 0.9565 | 0.9566 |
94
+ | 0.1034 | 5.69 | 2050 | 0.1511 | 0.9554 | 0.9555 |
95
+ | 0.1034 | 5.83 | 2100 | 0.1443 | 0.9565 | 0.9566 |
96
+ | 0.1034 | 5.97 | 2150 | 0.1526 | 0.9554 | 0.9554 |
97
+ | 0.0925 | 6.11 | 2200 | 0.1753 | 0.9540 | 0.9541 |
98
+ | 0.0925 | 6.25 | 2250 | 0.1503 | 0.9557 | 0.9557 |
99
+ | 0.0925 | 6.39 | 2300 | 0.1827 | 0.9518 | 0.9518 |
100
+ | 0.0925 | 6.53 | 2350 | 0.1486 | 0.9568 | 0.9568 |
101
+ | 0.0925 | 6.67 | 2400 | 0.1652 | 0.9568 | 0.9569 |
102
+ | 0.0925 | 6.81 | 2450 | 0.1544 | 0.9537 | 0.9538 |
103
+ | 0.0925 | 6.94 | 2500 | 0.1599 | 0.9551 | 0.9552 |
104
+ | 0.082 | 7.08 | 2550 | 0.1748 | 0.9568 | 0.9569 |
105
+ | 0.082 | 7.22 | 2600 | 0.1765 | 0.9582 | 0.9583 |
106
+ | 0.082 | 7.36 | 2650 | 0.1699 | 0.9568 | 0.9569 |
107
+ | 0.082 | 7.5 | 2700 | 0.1631 | 0.9563 | 0.9563 |
108
+ | 0.082 | 7.64 | 2750 | 0.1759 | 0.9602 | 0.9602 |
109
+ | 0.082 | 7.78 | 2800 | 0.1746 | 0.9565 | 0.9566 |
110
+ | 0.082 | 7.92 | 2850 | 0.1561 | 0.9568 | 0.9569 |
111
+ | 0.0742 | 8.06 | 2900 | 0.1721 | 0.9577 | 0.9577 |
112
+ | 0.0742 | 8.19 | 2950 | 0.1877 | 0.9563 | 0.9563 |
113
+ | 0.0742 | 8.33 | 3000 | 0.1896 | 0.9549 | 0.9549 |
114
+ | 0.0742 | 8.47 | 3050 | 0.1751 | 0.9577 | 0.9577 |
115
+ | 0.0742 | 8.61 | 3100 | 0.1812 | 0.9577 | 0.9577 |
116
+ | 0.0742 | 8.75 | 3150 | 0.1845 | 0.9577 | 0.9577 |
117
+ | 0.0742 | 8.89 | 3200 | 0.1844 | 0.9579 | 0.9580 |
118
+ | 0.0659 | 9.03 | 3250 | 0.1963 | 0.9571 | 0.9571 |
119
+ | 0.0659 | 9.17 | 3300 | 0.1861 | 0.9577 | 0.9577 |
120
+ | 0.0659 | 9.31 | 3350 | 0.1941 | 0.9585 | 0.9586 |
121
+ | 0.0659 | 9.44 | 3400 | 0.1900 | 0.9565 | 0.9566 |
122
+ | 0.0659 | 9.58 | 3450 | 0.1903 | 0.9565 | 0.9566 |
123
+ | 0.0659 | 9.72 | 3500 | 0.1836 | 0.9579 | 0.9580 |
124
+ | 0.0659 | 9.86 | 3550 | 0.1818 | 0.9565 | 0.9566 |
125
+ | 0.0631 | 10.0 | 3600 | 0.1817 | 0.9571 | 0.9572 |
126
+
127
+
128
+ ### Framework versions
129
+
130
+ - Transformers 4.39.3
131
+ - Pytorch 2.2.1+cu121
132
+ - Datasets 2.18.0
133
+ - Tokenizers 0.15.2
config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "papluca/xlm-roberta-base-language-detection",
3
+ "architectures": [
4
+ "XLMRobertaForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "classifier_dropout": null,
9
+ "eos_token_id": 2,
10
+ "hidden_act": "gelu",
11
+ "hidden_dropout_prob": 0.1,
12
+ "hidden_size": 768,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 3072,
15
+ "layer_norm_eps": 1e-05,
16
+ "max_position_embeddings": 514,
17
+ "model_type": "xlm-roberta",
18
+ "num_attention_heads": 12,
19
+ "num_hidden_layers": 12,
20
+ "output_past": true,
21
+ "pad_token_id": 1,
22
+ "position_embedding_type": "absolute",
23
+ "problem_type": "single_label_classification",
24
+ "torch_dtype": "float32",
25
+ "transformers_version": "4.39.3",
26
+ "type_vocab_size": 1,
27
+ "use_cache": true,
28
+ "vocab_size": 250002
29
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d39a2402f69f4e4b5ab8ad03863054b23d6a870b9374a46e180301388d12bf1e
3
+ size 1112205008
sentencepiece.bpe.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865
3
+ size 5069051
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "<unk>",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1cc44ad7faaeec47241864835473fd5403f2da94673f3f764a77ebcb0a803ec
3
+ size 17083009
tokenizer_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<pad>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "250001": {
36
+ "content": "<mask>",
37
+ "lstrip": true,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "bos_token": "<s>",
45
+ "clean_up_tokenization_spaces": true,
46
+ "cls_token": "<s>",
47
+ "eos_token": "</s>",
48
+ "mask_token": "<mask>",
49
+ "model_max_length": 512,
50
+ "pad_token": "<pad>",
51
+ "sep_token": "</s>",
52
+ "tokenizer_class": "XLMRobertaTokenizer",
53
+ "unk_token": "<unk>"
54
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e2a0d6445759851ea5d571cb315f99b0f27db9f667510f06e090fa06f64dc3ef
3
+ size 4856