daljeetsingh/sql_ft_t5small_kag

sql-training-1725691627

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0117

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.005
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.0561	0.0674	500	0.0401
0.0398	0.1348	1000	0.0298
0.035	0.2022	1500	0.0259
0.0312	0.2696	2000	0.0231
0.0244	0.3370	2500	0.0204
0.0264	0.4044	3000	0.0191
0.0295	0.4718	3500	0.0181
0.0227	0.5392	4000	0.0171
0.0245	0.6066	4500	0.0162
0.02	0.6739	5000	0.0153
0.0193	0.7413	5500	0.0148
0.0198	0.8087	6000	0.0142
0.0231	0.8761	6500	0.0139
0.0224	0.9435	7000	0.0134
0.0133	1.0109	7500	0.0131
0.0166	1.0783	8000	0.0129
0.0173	1.1457	8500	0.0126
0.0143	1.2131	9000	0.0124
0.0105	1.2805	9500	0.0123
0.0193	1.3479	10000	0.0122
0.0183	1.4153	10500	0.0120
0.0142	1.4827	11000	0.0119
0.0128	1.5501	11500	0.0118
0.0132	1.6175	12000	0.0118
0.0143	1.6849	12500	0.0117
0.015	1.7523	13000	0.0117
0.0161	1.8197	13500	0.0117
0.0132	1.8870	14000	0.0117
0.0119	1.9544	14500	0.0117

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1