End of training

README.md

@@ -12,14 +12,15 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/theubaada/huggingface/runs/fvqvjsw6)
 # pegasus-x-large-booksum-16k
 
 This model is a fine-tuned version of [ubaada/pegasus-x-large-booksum-16k](https://huggingface.co/ubaada/pegasus-x-large-booksum-16k) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.9460
-- Rouge1: 0.3363
-- Rouge2: 0.0569
-- Rougel: 0.1490
+- Loss: 1.8948
+- Rouge1: 0.3044
+- Rouge2: 0.0517
+- Rougel: 0.1398
 
 ## Model description
 
@@ -38,36 +39,26 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 4e-05
+- learning_rate: 8e-05
 - train_batch_size: 8
 - eval_batch_size: 1
 - seed: 42
-- distributed_type: multi-GPU
-- num_devices: 2
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 64
-- total_eval_batch_size: 2
+- total_train_batch_size: 32
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 8
+- num_epochs: 1
 
 ### Training results
 
 | Training Loss | Epoch  | Step | Validation Loss | Rouge1 | Rouge2 | Rougel |
 |:-------------:|:------:|:----:|:---------------:|:------:|:------:|:------:|
-| 1.3846        | 0.9992 | 314  | 1.9879          | 0.2983 | 0.0463 | 0.1367 |
-| 1.1051        | 1.9992 | 628  | 1.9696          | 0.3354 | 0.0549 | 0.1373 |
-| 1.0564        | 2.9992 | 942  | 1.9652          | 0.3215 | 0.0526 | 0.1427 |
-| 1.0657        | 3.9992 | 1256 | 1.9545          | 0.3370 | 0.0579 | 0.1424 |
-| 0.9694        | 4.9984 | 1570 | 1.9517          | 0.3766 | 0.0721 | 0.1532 |
-| 1.0343        | 5.9976 | 1884 | 1.9474          | 0.3558 | 0.0638 | 0.1488 |
-| 1.3146        | 7.0    | 2199 | 1.9475          | 0.3443 | 0.0568 | 0.1440 |
-| 0.9008        | 7.9960 | 2512 | 1.9460          | 0.3363 | 0.0569 | 0.1490 |
+| 1.3001        | 0.9992 | 314  | 1.8948          | 0.3044 | 0.0517 | 0.1398 |
 
 
 ### Framework versions
 
-- Transformers 4.40.2
+- Transformers 4.41.0
 - Pytorch 2.2.0
 - Datasets 2.19.1
 - Tokenizers 0.19.1

config.json

@@ -54,7 +54,7 @@
   "stagger_local_blocks": true,
   "static_position_embeddings": true,
   "torch_dtype": "float32",
-  "transformers_version": "4.40.2",
+  "transformers_version": "4.41.0",
   "use_cache": true,
   "vocab_size": 96103
 }

generation_config.json

@@ -8,5 +8,5 @@
   "num_beams": 5,
   "pad_token_id": 0,
   "repetition_penalty": 2.0,
-  "transformers_version": "4.40.2"
+  "transformers_version": "4.41.0"
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4312f998cc885aa9d6c2e0290dd48340b2cc3f5b4f024ec175e879aef768c4d8
+oid sha256:ce98531bc23515683731dfeb0728868c4635ee09df0b7311be52edc52a4d5fb9
 size 2274730128

tokenizer_config.json

@@ -959,7 +959,7 @@
   "mask_token": "<mask_2>",
   "mask_token_sent": "<mask_1>",
   "max_length": 16384,
-  "model_max_length": 12288,
+  "model_max_length": 16384,
   "offset": 103,
   "pad_to_multiple_of": null,
   "pad_token": "<pad>",

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2d33237e5eb3e663b28240a6df4c60ba0a3c0b98eb915e78a21500839804f32d
-size 6776
+oid sha256:9d7a58e5c136fc96f10523fd719d0eb94df70f3e8ae510a25cb6296f2fed8d8e
+size 6968