cartesinus
commited on
Commit
•
ec782d9
1
Parent(s):
bcfa1ab
Update README.md
Browse files
README.md
CHANGED
@@ -23,23 +23,26 @@ should probably proofread and complete it, then remove this comment. -->
|
|
23 |
|
24 |
# iva_mt_wslot-m2m100_418M-0.1.0 en-pl
|
25 |
|
26 |
-
This model is a fine-tuned version of [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M) on the
|
|
|
27 |
|
28 |
-
|
29 |
-
It achieves the following results:
|
30 |
1) On the test set (iva_mt):
|
31 |
- BLEU (plain text): 39.1560
|
32 |
- BLEU (with slots): 63.8767
|
33 |
-
-
|
|
|
|
|
|
|
34 |
|
35 |
2) WMT20 (en2pl):
|
36 |
-
- BLEU (lowercased, tags removed): 15.0863
|
37 |
- BLEU for baseline m2m100-418M (plain text): 20.2750
|
|
|
|
|
38 |
|
39 |
-
|
40 |
-
- BLEU: 61.6249
|
41 |
|
42 |
-
|
43 |
- BLEU (plain text): 70.5597
|
44 |
- BLEU (with slots): 93.8200
|
45 |
|
|
|
23 |
|
24 |
# iva_mt_wslot-m2m100_418M-0.1.0 en-pl
|
25 |
|
26 |
+
This model is a fine-tuned version of [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M) on the
|
27 |
+
[iva_mt_wslot](https://huggingface.co/datasets/cartesinus/iva_mt_wslot) dataset. It achieves the following results:
|
28 |
|
|
|
|
|
29 |
1) On the test set (iva_mt):
|
30 |
- BLEU (plain text): 39.1560
|
31 |
- BLEU (with slots): 63.8767
|
32 |
+
- F1 score: (in preparation)
|
33 |
+
|
34 |
+
For reference BLEU for baseline m2m100-418M (plain text) was 21.9468. Second result (63.8767) is when tags are treated as "normal" words in sentence. Therefore that result
|
35 |
+
might be a bit misleading. Please refer to plain text results if you are not sure how to interpret them.
|
36 |
|
37 |
2) WMT20 (en2pl):
|
38 |
+
- BLEU (lowercased, tags removed): 15.0863
|
39 |
- BLEU for baseline m2m100-418M (plain text): 20.2750
|
40 |
+
|
41 |
+
For reference WMT20 submission systems in en-pl direction had between 25 and 30 BLEU
|
42 |
|
43 |
+
3) BLEU on the evaluation set (same as in below table 'Training results'): 61.6249
|
|
|
44 |
|
45 |
+
4) On the training set (to see how it adjusted to train):
|
46 |
- BLEU (plain text): 70.5597
|
47 |
- BLEU (with slots): 93.8200
|
48 |
|