indosum-lora-3 / README.md
apwic's picture
End of training
88e6cc0 verified
metadata
language:
  - id
license: apache-2.0
base_model: LazarusNLP/IndoNanoT5-base
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: indosum-lora-3
    results: []

indosum-lora-3

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5148
  • Rouge1: 72.9552
  • Rouge2: 66.0943
  • Rougel: 69.9638
  • Rougelsum: 72.0769
  • Gen Len: 102.96

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 16
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5.0

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
0.7809 1.0 892 0.5511 70.7387 63.5003 67.571 69.7824 100.4227
0.5981 2.0 1784 0.5352 71.407 64.2122 68.3064 70.4861 104.784
0.5542 3.0 2676 0.5363 72.3351 65.2788 69.2683 71.4248 102.4427
0.5238 4.0 3568 0.5148 72.8512 65.9167 69.8532 71.9591 102.4173
0.5034 5.0 4460 0.5105 72.6979 65.6731 69.6184 71.7604 102.2533

Framework versions

  • Transformers 4.40.2
  • Pytorch 2.3.1+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1