---
library_name: transformers
base_model: google/pegasus-cnn_dailymail
tags:
- generated_from_trainer
datasets:
- samsum
model-index:
- name: pegasus-samsum
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# pegasus-samsum

This model is a fine-tuned version of [google/pegasus-cnn_dailymail](https://huggingface.co/google/pegasus-cnn_dailymail) on the samsum dataset.
It achieves the following results on the evaluation set:
- Loss: 6.0105

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 16
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 1

### Training results

| Training Loss | Epoch  | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 10.6079       | 0.0109 | 10   | 10.7159         |
| 10.5191       | 0.0217 | 20   | 10.6397         |
| 10.5539       | 0.0326 | 30   | 10.5210         |
| 10.5117       | 0.0434 | 40   | 10.3669         |
| 10.3111       | 0.0543 | 50   | 10.1941         |
| 10.2          | 0.0652 | 60   | 10.0230         |
| 10.1121       | 0.0760 | 70   | 9.8584          |
| 10.0677       | 0.0869 | 80   | 9.7129          |
| 9.7897        | 0.0977 | 90   | 9.5781          |
| 9.744         | 0.1086 | 100  | 9.4538          |
| 9.533         | 0.1195 | 110  | 9.3431          |
| 9.5248        | 0.1303 | 120  | 9.2552          |
| 9.3331        | 0.1412 | 130  | 9.1575          |
| 9.2551        | 0.1520 | 140  | 9.0751          |
| 9.2382        | 0.1629 | 150  | 8.9993          |
| 9.1323        | 0.1738 | 160  | 8.9287          |
| 9.0574        | 0.1846 | 170  | 8.8628          |
| 9.0137        | 0.1955 | 180  | 8.7964          |
| 8.9097        | 0.2064 | 190  | 8.7340          |
| 8.8268        | 0.2172 | 200  | 8.6765          |
| 8.7116        | 0.2281 | 210  | 8.6173          |
| 8.7483        | 0.2389 | 220  | 8.5521          |
| 8.6252        | 0.2498 | 230  | 8.4884          |
| 8.5844        | 0.2607 | 240  | 8.4275          |
| 8.4614        | 0.2715 | 250  | 8.3626          |
| 8.4375        | 0.2824 | 260  | 8.2901          |
| 8.445         | 0.2932 | 270  | 8.2102          |
| 8.2966        | 0.3041 | 280  | 8.1135          |
| 8.0934        | 0.3150 | 290  | 8.0113          |
| 8.1551        | 0.3258 | 300  | 7.8961          |
| 8.0542        | 0.3367 | 310  | 7.7751          |
| 8.0183        | 0.3475 | 320  | 7.6504          |
| 7.9352        | 0.3584 | 330  | 7.5371          |
| 7.7615        | 0.3693 | 340  | 7.4084          |
| 7.6484        | 0.3801 | 350  | 7.2715          |
| 7.5845        | 0.3910 | 360  | 7.1340          |
| 7.4799        | 0.4018 | 370  | 7.0563          |
| 7.3388        | 0.4127 | 380  | 6.9495          |
| 7.1078        | 0.4236 | 390  | 6.8582          |
| 7.0819        | 0.4344 | 400  | 6.7707          |
| 7.0465        | 0.4453 | 410  | 6.6897          |
| 6.9038        | 0.4561 | 420  | 6.6184          |
| 6.9359        | 0.4670 | 430  | 6.5533          |
| 6.8038        | 0.4779 | 440  | 6.4962          |
| 6.8648        | 0.4887 | 450  | 6.4452          |
| 6.7589        | 0.4996 | 460  | 6.3966          |
| 6.6804        | 0.5105 | 470  | 6.3588          |
| 6.5603        | 0.5213 | 480  | 6.3247          |
| 6.6002        | 0.5322 | 490  | 6.2937          |
| 6.59          | 0.5430 | 500  | 6.2665          |
| 6.5103        | 0.5539 | 510  | 6.2434          |
| 6.4911        | 0.5648 | 520  | 6.2210          |
| 6.4606        | 0.5756 | 530  | 6.2054          |
| 6.5193        | 0.5865 | 540  | 6.1870          |
| 6.4794        | 0.5973 | 550  | 6.1724          |
| 6.4579        | 0.6082 | 560  | 6.1598          |
| 6.3855        | 0.6191 | 570  | 6.1482          |
| 6.3071        | 0.6299 | 580  | 6.1367          |
| 6.4043        | 0.6408 | 590  | 6.1279          |
| 6.354         | 0.6516 | 600  | 6.1188          |
| 6.4038        | 0.6625 | 610  | 6.1114          |
| 6.3475        | 0.6734 | 620  | 6.1008          |
| 6.257         | 0.6842 | 630  | 6.0958          |
| 6.4359        | 0.6951 | 640  | 6.0872          |
| 6.2238        | 0.7059 | 650  | 6.0820          |
| 6.3904        | 0.7168 | 660  | 6.0754          |
| 6.2488        | 0.7277 | 670  | 6.0706          |
| 6.2648        | 0.7385 | 680  | 6.0644          |
| 6.303         | 0.7494 | 690  | 6.0601          |
| 6.3133        | 0.7602 | 700  | 6.0553          |
| 6.3229        | 0.7711 | 710  | 6.0516          |
| 6.3165        | 0.7820 | 720  | 6.0469          |
| 6.3353        | 0.7928 | 730  | 6.0438          |
| 6.2581        | 0.8037 | 740  | 6.0391          |
| 6.2688        | 0.8146 | 750  | 6.0361          |
| 6.2193        | 0.8254 | 760  | 6.0342          |
| 6.2247        | 0.8363 | 770  | 6.0305          |
| 6.1711        | 0.8471 | 780  | 6.0284          |
| 6.3126        | 0.8580 | 790  | 6.0259          |
| 6.3182        | 0.8689 | 800  | 6.0239          |
| 6.2298        | 0.8797 | 810  | 6.0214          |
| 6.287         | 0.8906 | 820  | 6.0198          |
| 6.2472        | 0.9014 | 830  | 6.0181          |
| 6.205         | 0.9123 | 840  | 6.0165          |
| 6.2359        | 0.9232 | 850  | 6.0147          |
| 6.3013        | 0.9340 | 860  | 6.0135          |
| 6.2035        | 0.9449 | 870  | 6.0129          |
| 6.2529        | 0.9557 | 880  | 6.0122          |
| 6.2043        | 0.9666 | 890  | 6.0114          |
| 6.2785        | 0.9775 | 900  | 6.0110          |
| 6.3018        | 0.9883 | 910  | 6.0106          |
| 6.1616        | 0.9992 | 920  | 6.0105          |


### Framework versions

- Transformers 4.44.2
- Pytorch 2.5.0+cu121
- Datasets 3.1.0
- Tokenizers 0.19.1