flan-t5-small-coref

This model is a fine-tuned version of google/flan-t5-small on the winograd_wsc dataset.

The model was trained on the task of coreference resolution.

It achieves the following results on the evaluation set:

• Loss: 0.5656
• Rouge1: 0.906
• Rouge2: 0.8192
• Rougel: 0.9016
• Rougelsum: 0.9026
• Gen Len: 23.1724

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	16	1.0901	0.6849	0.561	0.6734	0.6746	18.4483
No log	2.0	32	0.9083	0.8512	0.7509	0.8438	0.8437	21.1379
No log	3.0	48	0.8132	0.8638	0.7728	0.8588	0.8595	21.8276
No log	4.0	64	0.7590	0.8786	0.7842	0.8744	0.876	22.2069
No log	5.0	80	0.7225	0.8846	0.7928	0.8805	0.8817	22.3793
No log	6.0	96	0.6920	0.886	0.7942	0.8821	0.8827	22.4483
No log	7.0	112	0.6660	0.8861	0.7922	0.8816	0.8827	22.5172
No log	8.0	128	0.6470	0.8879	0.7953	0.8836	0.8849	22.6897
No log	9.0	144	0.6318	0.8968	0.806	0.8923	0.8933	23.069
No log	10.0	160	0.6160	0.8968	0.806	0.8923	0.8933	23.069
No log	11.0	176	0.6055	0.9056	0.822	0.9014	0.9021	23.1724
No log	12.0	192	0.5962	0.9056	0.822	0.9014	0.9021	23.1724
No log	13.0	208	0.5884	0.9074	0.8246	0.9033	0.9042	23.2069
No log	14.0	224	0.5825	0.9049	0.8182	0.9005	0.9016	23.2414
No log	15.0	240	0.5769	0.9049	0.8182	0.9005	0.9016	23.2414
No log	16.0	256	0.5727	0.903	0.8132	0.8991	0.8997	23.1724
No log	17.0	272	0.5698	0.906	0.8192	0.9016	0.9026	23.1724
No log	18.0	288	0.5673	0.906	0.8192	0.9016	0.9026	23.1724
No log	19.0	304	0.5661	0.906	0.8192	0.9016	0.9026	23.1724
No log	20.0	320	0.5656	0.906	0.8192	0.9016	0.9026	23.1724

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.0+cu116
• Datasets 2.7.1
• Tokenizers 0.13.2