indosum-pt-pl5-0

This model is a fine-tuned version of LazarusNLP/IndoNanoT5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7467
• Rouge1: 53.1916
• Rouge2: 33.4
• Rougel: 48.1427
• Rougelsum: 51.4084
• Gen Len: 94.7093

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.3529	1.0	892	2.4031	38.6954	16.1544	31.8171	36.2119	95.0347
2.9862	2.0	1784	2.1683	43.9573	21.3807	37.3906	41.5485	92.3707
2.7929	3.0	2676	1.9973	44.5846	23.2132	38.5252	42.4754	100.268
2.6146	4.0	3568	1.8518	48.6507	27.6554	42.9432	46.628	98.7813
2.4702	5.0	4460	1.7467	51.1569	30.4226	45.5936	49.2619	96.5467

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1