cartesinus commited on
Commit
8b4fac5
1 Parent(s): ed874a5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -8
README.md CHANGED
@@ -3,7 +3,7 @@ license: mit
3
  tags:
4
  - generated_from_trainer
5
  datasets:
6
- - iva_mt_wslot
7
  metrics:
8
  - bleu
9
  model-index:
@@ -22,6 +22,10 @@ model-index:
22
  - name: Bleu
23
  type: bleu
24
  value: 72.5602
 
 
 
 
25
  ---
26
 
27
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -39,13 +43,37 @@ It achieves the following results on the evaluation set:
39
 
40
  More information needed
41
 
42
- ## Intended uses & limitations
43
-
44
- More information needed
45
-
46
- ## Training and evaluation data
47
-
48
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
 
50
  ## Training procedure
51
 
 
3
  tags:
4
  - generated_from_trainer
5
  datasets:
6
+ - cartesinus/iva_mt_wslot
7
  metrics:
8
  - bleu
9
  model-index:
 
22
  - name: Bleu
23
  type: bleu
24
  value: 72.5602
25
+ language:
26
+ - en
27
+ - fr
28
+ pipeline_tag: translation
29
  ---
30
 
31
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
43
 
44
  More information needed
45
 
46
+ ## How to use
47
+
48
+ First please make sure to install `pip install transformers`. First download model:
49
+
50
+ ```python
51
+ from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
52
+ import torch
53
+
54
+ def translate(input_text, lang):
55
+ input_ids = tokenizer(input_text, return_tensors="pt")
56
+ generated_tokens = model.generate(**input_ids, forced_bos_token_id=tokenizer.get_lang_id(lang))
57
+ return tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
58
+
59
+ model_name = "cartesinus/iva_mt_wslot-m2m100_418M-0.1.0-en-fr"
60
+ tokenizer = M2M100Tokenizer.from_pretrained(model_name, src_lang="en", tgt_lang="fr")
61
+ model = M2M100ForConditionalGeneration.from_pretrained(model_name)
62
+ ```
63
+
64
+ Then you can translate either plain text like this:
65
+ ```python
66
+ print(translate("set the temperature on my thermostat", "fr"))
67
+ ```
68
+ or you can translate with slot annotations that will be restored in tgt language:
69
+ ```python
70
+ print(translate("wake me up at <a>nine am<a> on <b>friday<b>", "fr"))
71
+ ```
72
+ Limitations of translation with slot transfer:
73
+ 1) Annotated words must be placed between semi-xml tags like this "this is \<a\>example\<a\>"
74
+ 2) There is no closing tag for example "\<\a\>" in the above example - this is done on purpose to omit problems with backslash escape
75
+ 3) If the sentence consists of more than one slot then simply use the next alphabet letter. For example "this is \<a\>example\<a\> with more than \<b\>one\<b\> slot"
76
+ 4) Please do not add space before the first or last annotated word because this particular model was trained this way and it most probably will lower its results
77
 
78
  ## Training procedure
79