---
tags:
- generated_from_trainer
datasets:
- govreport-summarization
model-index:
- name: Pegasus-x-base-govreport-12288-1024-numepoch-5
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Pegasus-x-base-govreport-12288-1024-numepoch-5

This model is a fine-tuned version of [google/pegasus-x-base](https://huggingface.co/google/pegasus-x-base) on the govreport-summarization dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6740



## Evaluation Score

For test dataset

**'ROUGE'**:  

{
'rouge1': 0.4861,  
'rouge2': 0.2067,  
'rougeL': 0.2446,  
'rougeLsum': 0.2444  
}


**'BERT_SCORE'**  
{'f1': 0.8551,  
'precision': 0.8583,  
'recall': 0.852  
}



## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 3.0173        | 0.07  | 20   | 2.6677          |
| 2.5674        | 0.15  | 40   | 2.2993          |
| 2.3013        | 0.22  | 60   | 2.1024          |
| 2.2145        | 0.29  | 80   | 1.9833          |
| 2.1191        | 0.37  | 100  | 1.9383          |
| 2.0709        | 0.44  | 120  | 1.8815          |
| 2.0287        | 0.51  | 140  | 1.8623          |
| 2.003         | 0.58  | 160  | 1.8467          |
| 1.9842        | 0.66  | 180  | 1.8314          |
| 1.9603        | 0.73  | 200  | 1.8307          |
| 1.9493        | 0.8   | 220  | 1.8157          |
| 1.9631        | 0.88  | 240  | 1.7919          |
| 1.9332        | 0.95  | 260  | 1.7919          |
| 1.9123        | 1.02  | 280  | 1.7836          |
| 1.887         | 1.1   | 300  | 1.7672          |
| 1.8743        | 1.17  | 320  | 1.7629          |
| 1.8412        | 1.24  | 340  | 1.7566          |
| 1.8508        | 1.32  | 360  | 1.7410          |
| 1.8564        | 1.39  | 380  | 1.7403          |
| 1.8686        | 1.46  | 400  | 1.7393          |
| 1.8881        | 1.53  | 420  | 1.7420          |
| 1.8629        | 1.61  | 440  | 1.7367          |
| 1.8683        | 1.68  | 460  | 1.7288          |
| 1.833         | 1.75  | 480  | 1.7300          |
| 1.8621        | 1.83  | 500  | 1.7208          |
| 1.8622        | 1.9   | 520  | 1.7211          |
| 1.8147        | 1.97  | 540  | 1.7158          |
| 1.8161        | 2.05  | 560  | 1.7117          |
| 1.8239        | 2.12  | 580  | 1.7090          |
| 1.8185        | 2.19  | 600  | 1.7100          |
| 1.8605        | 2.27  | 620  | 1.7057          |
| 1.7919        | 2.34  | 640  | 1.6996          |
| 1.8026        | 2.41  | 660  | 1.7012          |
| 1.7785        | 2.48  | 680  | 1.6980          |
| 1.8296        | 2.56  | 700  | 1.6941          |
| 1.802         | 2.63  | 720  | 1.6944          |
| 1.7783        | 2.7   | 740  | 1.6927          |
| 1.7998        | 2.78  | 760  | 1.6922          |
| 1.8128        | 2.85  | 780  | 1.6890          |
| 1.7762        | 2.92  | 800  | 1.6909          |
| 1.7631        | 3.0   | 820  | 1.6959          |
| 1.8191        | 3.07  | 840  | 1.6823          |
| 1.795         | 3.14  | 860  | 1.6873          |
| 1.7587        | 3.22  | 880  | 1.6850          |
| 1.8091        | 3.29  | 900  | 1.6828          |
| 1.7617        | 3.36  | 920  | 1.6860          |
| 1.7933        | 3.43  | 940  | 1.6796          |
| 1.8041        | 3.51  | 960  | 1.6805          |
| 1.7596        | 3.58  | 980  | 1.6855          |
| 1.7518        | 3.65  | 1000 | 1.6791          |
| 1.7384        | 3.73  | 1020 | 1.6795          |
| 1.7855        | 3.8   | 1040 | 1.6784          |
| 1.7938        | 3.87  | 1060 | 1.6780          |
| 1.7637        | 3.95  | 1080 | 1.6809          |
| 1.7914        | 4.02  | 1100 | 1.6779          |
| 1.7903        | 4.09  | 1120 | 1.6753          |
| 1.7874        | 4.17  | 1140 | 1.6745          |
| 1.7982        | 4.24  | 1160 | 1.6728          |
| 1.7709        | 4.31  | 1180 | 1.6761          |
| 1.7583        | 4.38  | 1200 | 1.6754          |
| 1.778         | 4.46  | 1220 | 1.6739          |
| 1.7526        | 4.53  | 1240 | 1.6746          |
| 1.7713        | 4.6   | 1260 | 1.6723          |
| 1.734         | 4.68  | 1280 | 1.6742          |
| 1.7498        | 4.75  | 1300 | 1.6737          |
| 1.751         | 4.82  | 1320 | 1.6730          |
| 1.7562        | 4.9   | 1340 | 1.6739          |
| 1.7549        | 4.97  | 1360 | 1.6740          |


### Framework versions

- Transformers 4.30.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3