oop-de-qag-flan-t5-base
This model is a fine-tuned version of google/flan-t5-base on the dataset LunaticTanuki/oop-de-qg-v1. It achieves the following results on the evaluation set:
- Loss: 2.1427
- Rouge1: 22.9468
- Rouge2: 9.8345
- Rougel: 21.0791
- Rougelsum: 21.0408
- Gen Len: 16.2656
Model description
The model generates a question based on a paragraph as input.
Intended uses & limitations
The model was trained on data specifically targeting questions regarding object-oriented programming, so it only performs reliable in related topics.
Training and evaluation data
The paragraph and questions were used from the training dataset and validation dataset: LunaticTanuki/oop-de-qg-v1
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 8
Training results
Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
---|---|---|---|---|---|---|---|---|
No log | 1.0 | 127 | 2.2696 | 21.3464 | 8.6013 | 19.8612 | 19.8612 | 15.4219 |
No log | 2.0 | 254 | 2.1758 | 17.8678 | 6.3308 | 16.6657 | 16.7294 | 16.0156 |
No log | 3.0 | 381 | 2.1854 | 20.5546 | 7.3444 | 18.5305 | 18.631 | 16.2812 |
1.784 | 4.0 | 508 | 2.1831 | 23.9898 | 10.4013 | 22.2099 | 22.3739 | 16.2188 |
1.784 | 5.0 | 635 | 2.1704 | 22.0357 | 8.4803 | 20.8237 | 20.841 | 16.1562 |
1.784 | 6.0 | 762 | 2.1553 | 24.0652 | 10.8264 | 22.056 | 22.1786 | 16.7031 |
1.784 | 7.0 | 889 | 2.1427 | 22.9468 | 9.8345 | 21.0791 | 21.0408 | 16.2656 |
1.4159 | 8.0 | 1016 | 2.1532 | 23.8573 | 10.3393 | 21.9539 | 21.9372 | 16.4531 |
Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
- Downloads last month
- 9
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for LunaticTanuki/oop-de-qg-flan-t5-base-v1
Base model
google/flan-t5-base