Vít Novotný
commited on
Commit
•
204c0b9
1
Parent(s):
6b90090
Update `README.md`
Browse files
README.md
CHANGED
@@ -8,10 +8,10 @@ datasets:
|
|
8 |
|
9 |
# MathBERTa base model
|
10 |
|
11 |
-
Pretrained model on English language using a masked language modeling
|
12 |
-
objective. It was developed for [the ARQMath-3 shared task evaluation][1]
|
13 |
-
CLEF 2022 and first released in [this repository][2]. This model is
|
14 |
-
it makes a difference between english and English.
|
15 |
|
16 |
[1]: https://www.cs.rit.edu/~dprl/ARQMath/
|
17 |
[2]: https://github.com/witiko/scm-at-arqmath3
|
@@ -26,8 +26,8 @@ Like RoBERTa, MathBERTa has been fine-tuned with the Masked language modeling
|
|
26 |
(MLM) objective. Taking a sentence, the model randomly masks 15% of the words
|
27 |
and math symbols in the input then run the entire masked sentence through the
|
28 |
model and has to predict the masked words and symbols. This way, the model
|
29 |
-
learns an inner representation of the English language and
|
30 |
-
|
31 |
|
32 |
[3]: https://huggingface.co/roberta-base
|
33 |
[7]: https://github.com/Witiko/scm-at-arqmath3/blob/main/02-train-tokenizers.ipynb
|
|
|
8 |
|
9 |
# MathBERTa base model
|
10 |
|
11 |
+
Pretrained model on English language and LaTeX using a masked language modeling
|
12 |
+
(MLM) objective. It was developed for [the ARQMath-3 shared task evaluation][1]
|
13 |
+
at CLEF 2022 and first released in [this repository][2]. This model is
|
14 |
+
case-sensitive: it makes a difference between english and English.
|
15 |
|
16 |
[1]: https://www.cs.rit.edu/~dprl/ARQMath/
|
17 |
[2]: https://github.com/witiko/scm-at-arqmath3
|
|
|
26 |
(MLM) objective. Taking a sentence, the model randomly masks 15% of the words
|
27 |
and math symbols in the input then run the entire masked sentence through the
|
28 |
model and has to predict the masked words and symbols. This way, the model
|
29 |
+
learns an inner representation of the English language and LaTeX that can then
|
30 |
+
be used to extract features useful for downstream tasks.
|
31 |
|
32 |
[3]: https://huggingface.co/roberta-base
|
33 |
[7]: https://github.com/Witiko/scm-at-arqmath3/blob/main/02-train-tokenizers.ipynb
|