Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-4.0
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
- ar
|
6 |
+
- tl
|
7 |
+
- es
|
8 |
+
- ru
|
9 |
+
- ko
|
10 |
+
- ja
|
11 |
+
- ms
|
12 |
+
- id
|
13 |
+
- he
|
14 |
+
- hr
|
15 |
+
- pl
|
16 |
+
- pt
|
17 |
+
- af
|
18 |
+
- am
|
19 |
+
- az
|
20 |
+
- et
|
21 |
+
- eo
|
22 |
+
- th
|
23 |
+
- te
|
24 |
+
- ta
|
25 |
+
- tr
|
26 |
+
- uk
|
27 |
+
- ur
|
28 |
+
- uz
|
29 |
+
- vi
|
30 |
+
- ro
|
31 |
+
- km
|
32 |
+
- da
|
33 |
+
- de
|
34 |
+
- fi
|
35 |
+
- fa
|
36 |
+
- fr
|
37 |
+
- ga
|
38 |
+
- hu
|
39 |
+
- ky
|
40 |
+
- sa
|
41 |
+
tags:
|
42 |
+
- multilingual
|
43 |
+
- bert
|
44 |
+
- roberta
|
45 |
+
- xlmr
|
46 |
+
- bm
|
47 |
+
---
|
48 |
+
## Model type: Transformer-based masked language model
|
49 |
+
## Training data: No additional pretraining, merges two existing models
|
50 |
+
## Languages: 100+ languages
|
51 |
+
# Architecture:
|
52 |
+
- Base architectures:
|
53 |
+
- XLM-RoBERTa base (multilingual)
|
54 |
+
- BERT base cased (multilingual)
|
55 |
+
|
56 |
+
## Custom merging technique to combine weights from both base models into one unified model
|
57 |
+
|
58 |
+
# Mask token: <mask>
|