---
tags:
- generated_from_trainer
datasets:
- govreport-summarization
model-index:
- name: Pegasus-x-base-govreport-12288-1024-numepoch-10
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Pegasus-x-base-govreport-12288-1024-numepoch-10

This model is a fine-tuned version of [google/pegasus-x-base](https://huggingface.co/google/pegasus-x-base) on the govreport-summarization dataset.
It achieves the following results on the evaluation set:
- Loss: 1.6234

## Model description

More information needed

## Evaluation Score

**'ROUGE'**:  

{
'rouge1': 0.5012,  
'rouge2': 0.2205,  
'rougeL': 0.2552,  
'rougeLsum': 0.2554  
}


**'BERT_SCORE'**  
{'f1': 0.859,  
'precision': 0.8619,  
'recall': 0.8563  
}



## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 1
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 2.1149        | 0.37  | 100  | 1.9237          |
| 1.9545        | 0.73  | 200  | 1.8380          |
| 1.8835        | 1.1   | 300  | 1.7574          |
| 1.862         | 1.46  | 400  | 1.7305          |
| 1.8536        | 1.83  | 500  | 1.7100          |
| 1.8062        | 2.19  | 600  | 1.6944          |
| 1.8161        | 2.56  | 700  | 1.6882          |
| 1.7611        | 2.92  | 800  | 1.6803          |
| 1.7878        | 3.29  | 900  | 1.6671          |
| 1.7299        | 3.65  | 1000 | 1.6599          |
| 1.7636        | 4.02  | 1100 | 1.6558          |
| 1.7262        | 4.38  | 1200 | 1.6547          |
| 1.715         | 4.75  | 1300 | 1.6437          |
| 1.7178        | 5.12  | 1400 | 1.6445          |
| 1.7163        | 5.48  | 1500 | 1.6386          |
| 1.7367        | 5.85  | 1600 | 1.6364          |
| 1.7114        | 6.21  | 1700 | 1.6365          |
| 1.6452        | 6.58  | 1800 | 1.6309          |
| 1.7251        | 6.94  | 1900 | 1.6301          |
| 1.6726        | 7.31  | 2000 | 1.6305          |
| 1.7104        | 7.67  | 2100 | 1.6285          |
| 1.6739        | 8.04  | 2200 | 1.6252          |
| 1.7082        | 8.4   | 2300 | 1.6246          |
| 1.6888        | 8.77  | 2400 | 1.6244          |
| 1.6609        | 9.13  | 2500 | 1.6256          |
| 1.6707        | 9.5   | 2600 | 1.6241          |
| 1.669         | 9.86  | 2700 | 1.6234          |


### Framework versions

- Transformers 4.30.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3