---
base_model: silmi224/finetune-led-35000
tags:
- summarization
- generated_from_trainer
metrics:
- rouge
model-index:
- name: exp2-led-risalah_data_v1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# exp2-led-risalah_data_v1

This model is a fine-tuned version of [silmi224/finetune-led-35000](https://huggingface.co/silmi224/finetune-led-35000) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.5716
- Rouge1: 19.6052
- Rouge2: 10.044
- Rougel: 14.481
- Rougelsum: 18.8794

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 150
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|
| 3.3238        | 1.0   | 10   | 2.7939          | 8.2919  | 2.3936  | 6.4452  | 7.8858    |
| 3.0941        | 2.0   | 20   | 2.5393          | 8.884   | 2.3029  | 6.6763  | 8.2192    |
| 2.7749        | 3.0   | 30   | 2.2953          | 11.5346 | 3.61    | 7.9843  | 10.5072   |
| 2.5095        | 4.0   | 40   | 2.1271          | 13.3139 | 4.2267  | 9.3106  | 11.9731   |
| 2.3044        | 5.0   | 50   | 2.0140          | 14.8318 | 5.4652  | 10.4052 | 13.8404   |
| 2.1532        | 6.0   | 60   | 1.9280          | 15.6855 | 6.4587  | 10.7669 | 14.5093   |
| 2.032         | 7.0   | 70   | 1.8598          | 14.9367 | 5.7627  | 10.481  | 14.0571   |
| 1.9376        | 8.0   | 80   | 1.8049          | 14.8866 | 6.1106  | 10.0165 | 14.4686   |
| 1.8459        | 9.0   | 90   | 1.7491          | 13.6909 | 5.6398  | 9.2128  | 12.9399   |
| 1.7765        | 10.0  | 100  | 1.7213          | 16.7363 | 7.2146  | 11.2988 | 16.0402   |
| 1.704         | 11.0  | 110  | 1.6857          | 18.4687 | 8.7089  | 12.9138 | 17.8621   |
| 1.6542        | 12.0  | 120  | 1.6610          | 19.2238 | 8.9265  | 13.1614 | 17.6696   |
| 1.5957        | 13.0  | 130  | 1.6335          | 19.6057 | 9.8766  | 13.6908 | 18.6659   |
| 1.5413        | 14.0  | 140  | 1.6145          | 19.2875 | 9.7272  | 14.3241 | 17.7305   |
| 1.496         | 15.0  | 150  | 1.6232          | 18.1669 | 8.857   | 13.5735 | 17.1252   |
| 1.4535        | 16.0  | 160  | 1.6036          | 19.3501 | 10.1008 | 14.5871 | 18.4397   |
| 1.4204        | 17.0  | 170  | 1.5954          | 19.4201 | 10.3577 | 14.2019 | 18.4312   |
| 1.3829        | 18.0  | 180  | 1.5794          | 18.4944 | 9.4098  | 13.9    | 17.3891   |
| 1.3535        | 19.0  | 190  | 1.5814          | 19.9886 | 11.1416 | 15.0161 | 19.122    |
| 1.3328        | 20.0  | 200  | 1.5758          | 20.2011 | 10.5645 | 14.7218 | 19.1219   |
| 1.3063        | 21.0  | 210  | 1.5722          | 20.7308 | 10.834  | 15.3016 | 19.8805   |
| 1.2858        | 22.0  | 220  | 1.5745          | 19.648  | 10.77   | 14.0294 | 19.0395   |
| 1.2726        | 23.0  | 230  | 1.5651          | 20.4129 | 10.8196 | 15.0054 | 19.7253   |
| 1.2557        | 24.0  | 240  | 1.5709          | 18.6308 | 9.3525  | 13.8142 | 18.1621   |
| 1.2456        | 25.0  | 250  | 1.5659          | 19.6106 | 10.4499 | 14.4439 | 18.9271   |
| 1.233         | 26.0  | 260  | 1.5702          | 19.1583 | 9.7391  | 14.1738 | 18.5077   |
| 1.2267        | 27.0  | 270  | 1.5651          | 18.7654 | 9.8637  | 13.7809 | 18.2034   |
| 1.2203        | 28.0  | 280  | 1.5703          | 19.9698 | 10.4741 | 14.3559 | 19.389    |
| 1.2147        | 29.0  | 290  | 1.5739          | 19.9054 | 10.0052 | 14.6427 | 19.2278   |
| 1.2124        | 30.0  | 300  | 1.5716          | 19.6052 | 10.044  | 14.481  | 18.8794   |


### Framework versions

- Transformers 4.41.2
- Pytorch 2.1.2
- Datasets 2.19.2
- Tokenizers 0.19.1