fine-tuned-DatasetQAS-Squad-ID-with-indobert-large-p2-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indobenchmark/indobert-large-p2 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5867
• Exact Match: 47.7296
• F1: 64.3850

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 128
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
1.8839	0.5	463	1.7873	39.9512	56.0205
1.6682	1.0	926	1.6243	44.2651	60.9585
1.5129	1.5	1389	1.5722	45.6609	61.7661
1.4634	2.0	1852	1.5185	47.1493	63.5348
1.3128	2.5	2315	1.5212	46.9475	63.4277
1.323	3.0	2778	1.5052	47.6118	64.2591
1.1824	3.5	3241	1.5352	47.5950	64.2896
1.2013	4.0	3704	1.5302	47.9566	64.5453
1.0842	4.5	4167	1.5678	47.5362	64.2029
1.0811	5.0	4630	1.5590	47.7632	64.1309
1.0138	5.5	5093	1.5867	47.7296	64.3850

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2