Hugging Face's logo

lewtun
/
mt5-small-finetuned-mlsum

Text2Text Generation
Transformers
PyTorch
TensorBoard
mt5
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Metrics Community
1
lewtun HF staff commited on
Commit
4340696
•
1 Parent(s): 6d9ad0c

add model

Browse files
Files changed (20) hide show
  1. .gitignore +1 -0
  2. README.md +77 -0
  3. config.json +28 -0
  4. pytorch_model.bin +3 -0
  5. runs/Sep25_08-46-57_4fcb35322c42/1632559692.9582436/events.out.tfevents.1632559692.4fcb35322c42.149.1 +3 -0
  6. runs/Sep25_08-46-57_4fcb35322c42/events.out.tfevents.1632559692.4fcb35322c42.149.0 +3 -0
  7. runs/Sep25_08-49-31_4fcb35322c42/1632559794.5055263/events.out.tfevents.1632559794.4fcb35322c42.679.1 +3 -0
  8. runs/Sep25_08-49-31_4fcb35322c42/events.out.tfevents.1632559794.4fcb35322c42.679.0 +3 -0
  9. runs/Sep25_09-09-01_4fcb35322c42/1632560953.3960927/events.out.tfevents.1632560953.4fcb35322c42.679.3 +3 -0
  10. runs/Sep25_09-09-01_4fcb35322c42/events.out.tfevents.1632560953.4fcb35322c42.679.2 +3 -0
  11. runs/Sep25_09-09-27_4fcb35322c42/1632560978.0959616/events.out.tfevents.1632560978.4fcb35322c42.679.5 +3 -0
  12. runs/Sep25_09-09-27_4fcb35322c42/events.out.tfevents.1632560978.4fcb35322c42.679.4 +3 -0
  13. runs/Sep25_09-16-00_4fcb35322c42/1632561511.1224031/events.out.tfevents.1632561511.4fcb35322c42.1113.2 +3 -0
  14. runs/Sep25_09-16-00_4fcb35322c42/events.out.tfevents.1632561488.4fcb35322c42.1113.0 +3 -0
  15. runs/Sep25_09-16-00_4fcb35322c42/events.out.tfevents.1632561511.4fcb35322c42.1113.1 +3 -0
  16. special_tokens_map.json +1 -0
  17. spiece.model +3 -0
  18. tokenizer.json +0 -0
  19. tokenizer_config.json +1 -0
  20. training_args.bin +3 -0
.gitignore ADDED
@@ -0,0 +1 @@
 
 
1
+ checkpoint-*/
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - generated_from_trainer
5
+ datasets:
6
+ - mlsum
7
+ metrics:
8
+ - rouge
9
+ model-index:
10
+ - name: mt5-small-finetuned-mlsum
11
+ results:
12
+ - task:
13
+ name: Sequence-to-sequence Language Modeling
14
+ type: text2text-generation
15
+ dataset:
16
+ name: mlsum
17
+ type: mlsum
18
+ args: es
19
+ metrics:
20
+ - name: Rouge1
21
+ type: rouge
22
+ value: 1.1475
23
+ ---
24
+
25
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
26
+ should probably proofread and complete it, then remove this comment. -->
27
+
28
+ # mt5-small-finetuned-mlsum
29
+
30
+ This model is a fine-tuned version of [google/mt5-small](https://huggingface.co/google/mt5-small) on the mlsum dataset.
31
+ It achieves the following results on the evaluation set:
32
+ - Loss: nan
33
+ - Rouge1: 1.1475
34
+ - Rouge2: 0.1284
35
+ - Rougel: 1.0634
36
+ - Rougelsum: 1.0778
37
+ - Gen Len: 3.7939
38
+
39
+ ## Model description
40
+
41
+ More information needed
42
+
43
+ ## Intended uses & limitations
44
+
45
+ More information needed
46
+
47
+ ## Training and evaluation data
48
+
49
+ More information needed
50
+
51
+ ## Training procedure
52
+
53
+ ### Training hyperparameters
54
+
55
+ The following hyperparameters were used during training:
56
+ - learning_rate: 2e-05
57
+ - train_batch_size: 8
58
+ - eval_batch_size: 8
59
+ - seed: 42
60
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
61
+ - lr_scheduler_type: linear
62
+ - num_epochs: 1
63
+ - mixed_precision_training: Native AMP
64
+
65
+ ### Training results
66
+
67
+ | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
68
+ |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
69
+ | nan | 1.0 | 808 | nan | 1.1475 | 0.1284 | 1.0634 | 1.0778 | 3.7939 |
70
+
71
+
72
+ ### Framework versions
73
+
74
+ - Transformers 4.10.3
75
+ - Pytorch 1.9.0+cu102
76
+ - Datasets 1.12.1
77
+ - Tokenizers 0.10.3
config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "google/mt5-small",
3
+ "architectures": [
4
+ "MT5ForConditionalGeneration"
5
+ ],
6
+ "d_ff": 1024,
7
+ "d_kv": 64,
8
+ "d_model": 512,
9
+ "decoder_start_token_id": 0,
10
+ "dropout_rate": 0.1,
11
+ "eos_token_id": 1,
12
+ "feed_forward_proj": "gated-gelu",
13
+ "initializer_factor": 1.0,
14
+ "is_encoder_decoder": true,
15
+ "layer_norm_epsilon": 1e-06,
16
+ "model_type": "mt5",
17
+ "num_decoder_layers": 8,
18
+ "num_heads": 6,
19
+ "num_layers": 8,
20
+ "pad_token_id": 0,
21
+ "relative_attention_num_buckets": 32,
22
+ "tie_word_embeddings": false,
23
+ "tokenizer_class": "T5Tokenizer",
24
+ "torch_dtype": "float32",
25
+ "transformers_version": "4.10.3",
26
+ "use_cache": true,
27
+ "vocab_size": 250112
28
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:672b266857da449c8e1759ff96da0d8e7dc4402f0722687fee84220d436fe544
3
+ size 1200770885
runs/Sep25_08-46-57_4fcb35322c42/1632559692.9582436/events.out.tfevents.1632559692.4fcb35322c42.149.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75319c390510323ae2704fbdd2c0e652c19545f718497b94ebf7a519ab7fe9a4
3
+ size 4466
runs/Sep25_08-46-57_4fcb35322c42/events.out.tfevents.1632559692.4fcb35322c42.149.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb2f1a093f04dade2dd1d5871ac29a65c4a4a9260625239982bd4014ef67ae41
3
+ size 3129
runs/Sep25_08-49-31_4fcb35322c42/1632559794.5055263/events.out.tfevents.1632559794.4fcb35322c42.679.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b1df951bb8534eecacfa516aebe97876e5f0d0be4bda32fb261d686aa72a955
3
+ size 4466
runs/Sep25_08-49-31_4fcb35322c42/events.out.tfevents.1632559794.4fcb35322c42.679.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a55bb140fb4f834d29fe54d79e2294491c8e4ca457ed85fd652995f35eaab8e
3
+ size 3127
runs/Sep25_09-09-01_4fcb35322c42/1632560953.3960927/events.out.tfevents.1632560953.4fcb35322c42.679.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d416aebbcc6ba4e5ad6d3d2231ac3a61489f6533da1adf11b90220f56767f9a6
3
+ size 4466
runs/Sep25_09-09-01_4fcb35322c42/events.out.tfevents.1632560953.4fcb35322c42.679.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cbab113a07480e8fcfbe0da766c495fe071399030581b604af503b5a30aa7d53
3
+ size 3127
runs/Sep25_09-09-27_4fcb35322c42/1632560978.0959616/events.out.tfevents.1632560978.4fcb35322c42.679.5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6b0b34e01296abb7ccaffb22b1dbce17f770b875f5490b8d1ba3cb5cd330757
3
+ size 4466
runs/Sep25_09-09-27_4fcb35322c42/events.out.tfevents.1632560978.4fcb35322c42.679.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73c8e383cad28c099cc0c2216be3223fb3512fc70e11e49e1598b68ad81758ac
3
+ size 3129
runs/Sep25_09-16-00_4fcb35322c42/1632561511.1224031/events.out.tfevents.1632561511.4fcb35322c42.1113.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3bb0dddea3e382af546333bc9336ef9eaa15c1534be6825a00dccf221448d6bd
3
+ size 4466
runs/Sep25_09-16-00_4fcb35322c42/events.out.tfevents.1632561488.4fcb35322c42.1113.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7824d2dd9f8a9850510eca23490154e6320ece8b9e6b01a92ad5d6f852c3b46
3
+ size 488
runs/Sep25_09-16-00_4fcb35322c42/events.out.tfevents.1632561511.4fcb35322c42.1113.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62b2f6fc1d06f462892f1b745a3e14bf98079b3c65a82053cb49c58bb6eaae3c
3
+ size 4163
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}
spiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ef78f86560d809067d12bac6c09f19a462cb3af3f54d2b8acbba26e1433125d6
3
+ size 4309802
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 0, "additional_special_tokens": null, "special_tokens_map_file": "/root/.cache/huggingface/transformers/685ac0ca8568ec593a48b61b0a3c272beee9bc194a3c7241d15dcadb5f875e53.f76030f3ec1b96a8199b2593390c610e76ca8028ef3d24680000619ffb646276", "name_or_path": "google/mt5-small", "sp_model_kwargs": {}, "tokenizer_class": "T5Tokenizer"}
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dcd10eac0bb99f9d34c3dff15be732f29b61243d62f06ce127ccc9b8be1e3266
3
+ size 2799