LunaticTanuki/oop-de-qg-flan-t5-base-v1

oop-de-qag-flan-t5-base

This model is a fine-tuned version of google/flan-t5-base on the dataset LunaticTanuki/oop-de-qg-v1. It achieves the following results on the evaluation set:

• Loss: 2.1427
• Rouge1: 22.9468
• Rouge2: 9.8345
• Rougel: 21.0791
• Rougelsum: 21.0408
• Gen Len: 16.2656

Model description

The model generates a question based on a paragraph as input.

Intended uses & limitations

The model was trained on data specifically targeting questions regarding object-oriented programming, so it only performs reliable in related topics.

Training and evaluation data

The paragraph and questions were used from the training dataset and validation dataset: LunaticTanuki/oop-de-qg-v1

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	127	2.2696	21.3464	8.6013	19.8612	19.8612	15.4219
No log	2.0	254	2.1758	17.8678	6.3308	16.6657	16.7294	16.0156
No log	3.0	381	2.1854	20.5546	7.3444	18.5305	18.631	16.2812
1.784	4.0	508	2.1831	23.9898	10.4013	22.2099	22.3739	16.2188
1.784	5.0	635	2.1704	22.0357	8.4803	20.8237	20.841	16.1562
1.784	6.0	762	2.1553	24.0652	10.8264	22.056	22.1786	16.7031
1.784	7.0	889	2.1427	22.9468	9.8345	21.0791	21.0408	16.2656
1.4159	8.0	1016	2.1532	23.8573	10.3393	21.9539	21.9372	16.4531

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0