cartesinus commited on
Commit
ec782d9
1 Parent(s): bcfa1ab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -8
README.md CHANGED
@@ -23,23 +23,26 @@ should probably proofread and complete it, then remove this comment. -->
23
 
24
  # iva_mt_wslot-m2m100_418M-0.1.0 en-pl
25
 
26
- This model is a fine-tuned version of [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M) on the [iva_mt_wslot](https://huggingface.co/datasets/cartesinus/iva_mt_wslot) dataset.
 
27
 
28
-
29
- It achieves the following results:
30
  1) On the test set (iva_mt):
31
  - BLEU (plain text): 39.1560
32
  - BLEU (with slots): 63.8767
33
- - BLEU for baseline m2m100-418M (plain text): TBD
 
 
 
34
 
35
  2) WMT20 (en2pl):
36
- - BLEU (lowercased, tags removed): 15.0863 (for reference WMT20 submission systems in en-pl direction had between 25 and 30 bleu)
37
  - BLEU for baseline m2m100-418M (plain text): 20.2750
 
 
38
 
39
- 4) On the evaluation set:
40
- - BLEU: 61.6249
41
 
42
- 3) On the training set (to see how it adjusted to train):
43
  - BLEU (plain text): 70.5597
44
  - BLEU (with slots): 93.8200
45
 
 
23
 
24
  # iva_mt_wslot-m2m100_418M-0.1.0 en-pl
25
 
26
+ This model is a fine-tuned version of [facebook/m2m100_418M](https://huggingface.co/facebook/m2m100_418M) on the
27
+ [iva_mt_wslot](https://huggingface.co/datasets/cartesinus/iva_mt_wslot) dataset. It achieves the following results:
28
 
 
 
29
  1) On the test set (iva_mt):
30
  - BLEU (plain text): 39.1560
31
  - BLEU (with slots): 63.8767
32
+ - F1 score: (in preparation)
33
+
34
+ For reference BLEU for baseline m2m100-418M (plain text) was 21.9468. Second result (63.8767) is when tags are treated as "normal" words in sentence. Therefore that result
35
+ might be a bit misleading. Please refer to plain text results if you are not sure how to interpret them.
36
 
37
  2) WMT20 (en2pl):
38
+ - BLEU (lowercased, tags removed): 15.0863
39
  - BLEU for baseline m2m100-418M (plain text): 20.2750
40
+
41
+ For reference WMT20 submission systems in en-pl direction had between 25 and 30 BLEU
42
 
43
+ 3) BLEU on the evaluation set (same as in below table 'Training results'): 61.6249
 
44
 
45
+ 4) On the training set (to see how it adjusted to train):
46
  - BLEU (plain text): 70.5597
47
  - BLEU (with slots): 93.8200
48