Hugging Face's logo

classla
/
xlm-r-bertic

Fill-Mask
Transformers
PyTorch
Croatian
Bosnian
Serbian
xlm-roberta
Inference Endpoints
Model card Files Community
1
5roop commited on
Commit
1c67627
•
1 Parent(s): 57a5384

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -24
README.md CHANGED
@@ -24,37 +24,37 @@ Mean F1 scores were used to evaluate performance.
24
  |:-----------------------------------------------------------------------|:--------|---------:|
25
  | **XLM-R-BERTić** (this model) | hr500k | 0.927 |
26
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | hr500k | 0.925 |
27
- | XLM-R-SloBERTić | hr500k | 0.923 |
28
- | XLM-Roberta-Large | hr500k | 0.919 |
29
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | hr500k | 0.918 |
30
- | XLM-Roberta-Base | hr500k | 0.903 |
31
 
32
  | system | dataset | F1 score |
33
  |:-----------------------------------------------------------------------|:---------|---------:|
34
- | XLM-R-SloBERTić | ReLDI-hr | 0.812 |
35
- | **XLM-R-BERTić** (this model) | ReLDI-hr | 0.809 |
36
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | ReLDI-hr | 0.794 |
37
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | ReLDI-hr | 0.792 |
38
- | XLM-Roberta-Large | ReLDI-hr | 0.791 |
39
- | XLM-Roberta-Base | ReLDI-hr | 0.763 |
40
 
41
  | system | dataset | F1 score |
42
  |:-----------------------------------------------------------------------|:-----------|---------:|
43
- | XLM-R-SloBERTić | SETimes.SR | 0.949 |
44
- | **XLM-R-BERTić** (this model) | SETimes.SR | 0.940 |
45
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | SETimes.SR | 0.936 |
46
- | XLM-Roberta-Large | SETimes.SR | 0.933 |
47
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | SETimes.SR | 0.922 |
48
- | XLM-Roberta-Base | SETimes.SR | 0.914 |
49
 
50
  | system | dataset | F1 score |
51
  |:-----------------------------------------------------------------------|:---------|---------:|
52
- | **XLM-R-BERTić** (this model) | ReLDI-sr | 0.841 |
53
- | XLM-R-SloBERTić | ReLDI-sr | 0.824 |
54
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | ReLDI-sr | 0.798 |
55
- | XLM-Roberta-Large | ReLDI-sr | 0.774 |
56
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | ReLDI-sr | 0.751 |
57
- | XLM-Roberta-Base | ReLDI-sr | 0.734 |
58
 
59
  ## Sentiment regression
60
 
@@ -65,11 +65,11 @@ The procedure is explained in greater detail in the dedicated [benchmarking repo
65
  |:-----------------------------------------------------------------------|:--------------------|:-------------------------|------:|
66
  | [xlm-r-parlasent](https://huggingface.co/classla/xlm-r-parlasent) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.615 |
67
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.612 |
68
- | XLM-R-SloBERTić | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.607 |
69
- | XLM-Roberta-Large | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.605 |
70
  | **XLM-R-BERTić** (this model) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.601 |
71
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.537 |
72
- | XLM-Roberta-Base | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.500 |
73
  | dummy (mean) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | -0.12 |
74
 
75
 
@@ -79,21 +79,21 @@ The procedure is explained in greater detail in the dedicated [benchmarking repo
79
  | system | dataset | Accuracy score |
80
  |:-----------------------------------------------------------------------|:--------|---------------:|
81
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | Copa-SR | 0.689 |
82
- | XLM-R-SloBERTić | Copa-SR | 0.665 |
83
  | **XLM-R-BERTić** (this model) | Copa-SR | 0.637 |
84
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | Copa-SR | 0.607 |
85
- | XLM-Roberta-Base | Copa-SR | 0.573 |
86
- | XLM-Roberta-Large | Copa-SR | 0.570 |
87
 
88
 
89
  | system | dataset | Accuracy score |
90
  |:-----------------------------------------------------------------------|:--------|---------------:|
91
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | Copa-HR | 0.669 |
92
- | XLM-R-SloBERTić | Copa-HR | 0.628 |
93
  | **XLM-R-BERTić** (this model) | Copa-HR | 0.635 |
94
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | Copa-HR | 0.669 |
95
- | XLM-Roberta-Base | Copa-HR | 0.585 |
96
- | XLM-Roberta-Large | Copa-HR | 0.571 |
97
 
98
 
99
 
 
24
  |:-----------------------------------------------------------------------|:--------|---------:|
25
  | **XLM-R-BERTić** (this model) | hr500k | 0.927 |
26
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | hr500k | 0.925 |
27
+ | [XLM-R-SloBERTić](https://huggingface.co/classla/xlm-r-slobertic) | hr500k | 0.923 |
28
+ | [XLM-Roberta-Large](https://huggingface.co/xlm-roberta-large) | hr500k | 0.919 |
29
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | hr500k | 0.918 |
30
+ | [XLM-Roberta-Base](https://huggingface.co/xlm-roberta-base) | hr500k | 0.903 |
31
 
32
  | system | dataset | F1 score |
33
  |:-----------------------------------------------------------------------|:---------|---------:|
34
+ | [XLM-R-SloBERTić](https://huggingface.co/classla/xlm-r-slobertic) | ReLDI-hr | 0.812 |
35
+ | **XLM-R-BERTić** (this model) | ReLDI-hr | 0.809 |
36
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | ReLDI-hr | 0.794 |
37
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | ReLDI-hr | 0.792 |
38
+ | [XLM-Roberta-Large](https://huggingface.co/xlm-roberta-large) | ReLDI-hr | 0.791 |
39
+ | [XLM-Roberta-Base](https://huggingface.co/xlm-roberta-base) | ReLDI-hr | 0.763 |
40
 
41
  | system | dataset | F1 score |
42
  |:-----------------------------------------------------------------------|:-----------|---------:|
43
+ | [XLM-R-SloBERTić](https://huggingface.co/classla/xlm-r-slobertic) | SETimes.SR | 0.949 |
44
+ | **XLM-R-BERTić** (this model) | SETimes.SR | 0.940 |
45
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | SETimes.SR | 0.936 |
46
+ | [XLM-Roberta-Large](https://huggingface.co/xlm-roberta-large) | SETimes.SR | 0.933 |
47
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | SETimes.SR | 0.922 |
48
+ | [XLM-Roberta-Base](https://huggingface.co/xlm-roberta-base) | SETimes.SR | 0.914 |
49
 
50
  | system | dataset | F1 score |
51
  |:-----------------------------------------------------------------------|:---------|---------:|
52
+ | **XLM-R-BERTić** (this model) | ReLDI-sr | 0.841 |
53
+ | [XLM-R-SloBERTić](https://huggingface.co/classla/xlm-r-slobertic) | ReLDI-sr | 0.824 |
54
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | ReLDI-sr | 0.798 |
55
+ | [XLM-Roberta-Large](https://huggingface.co/xlm-roberta-large) | ReLDI-sr | 0.774 |
56
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | ReLDI-sr | 0.751 |
57
+ | [XLM-Roberta-Base](https://huggingface.co/xlm-roberta-base) | ReLDI-sr | 0.734 |
58
 
59
  ## Sentiment regression
60
 
 
65
  |:-----------------------------------------------------------------------|:--------------------|:-------------------------|------:|
66
  | [xlm-r-parlasent](https://huggingface.co/classla/xlm-r-parlasent) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.615 |
67
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.612 |
68
+ | [XLM-R-SloBERTić](https://huggingface.co/classla/xlm-r-slobertic) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.607 |
69
+ | [XLM-Roberta-Large](https://huggingface.co/xlm-roberta-large) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.605 |
70
  | **XLM-R-BERTić** (this model) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.601 |
71
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.537 |
72
+ | [XLM-Roberta-Base](https://huggingface.co/xlm-roberta-base) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | 0.500 |
73
  | dummy (mean) | ParlaSent_BCS.jsonl | ParlaSent_BCS_test.jsonl | -0.12 |
74
 
75
 
 
79
  | system | dataset | Accuracy score |
80
  |:-----------------------------------------------------------------------|:--------|---------------:|
81
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | Copa-SR | 0.689 |
82
+ | [XLM-R-SloBERTić](https://huggingface.co/classla/xlm-r-slobertic) | Copa-SR | 0.665 |
83
  | **XLM-R-BERTić** (this model) | Copa-SR | 0.637 |
84
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | Copa-SR | 0.607 |
85
+ | [XLM-Roberta-Base](https://huggingface.co/xlm-roberta-base) | Copa-SR | 0.573 |
86
+ | [XLM-Roberta-Large](https://huggingface.co/xlm-roberta-large) | Copa-SR | 0.570 |
87
 
88
 
89
  | system | dataset | Accuracy score |
90
  |:-----------------------------------------------------------------------|:--------|---------------:|
91
  | [BERTić](https://huggingface.co/classla/bcms-bertic) | Copa-HR | 0.669 |
92
+ | [XLM-R-SloBERTić](https://huggingface.co/classla/xlm-r-slobertic) | Copa-HR | 0.628 |
93
  | **XLM-R-BERTić** (this model) | Copa-HR | 0.635 |
94
  | [crosloengual-bert](https://huggingface.co/EMBEDDIA/crosloengual-bert) | Copa-HR | 0.669 |
95
+ | [XLM-Roberta-Base](https://huggingface.co/xlm-roberta-base) | Copa-HR | 0.585 |
96
+ | [XLM-Roberta-Large](https://huggingface.co/xlm-roberta-large) | Copa-HR | 0.571 |
97
 
98
 
99