TenaliAI-Banking-v1

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4832

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
1.0141	1.0	3229	0.8505
0.5894	2.0	6458	0.5827
0.505	3.0	9687	0.5536
0.4528	4.0	12916	0.5003
0.4438	5.0	16145	0.4981
0.4142	6.0	19374	0.4867
0.4055	7.0	22603	0.4881
0.3754	8.0	25832	0.4858
0.3923	9.0	29061	0.4877
0.3644	10.0	32290	0.4845
0.375	11.0	35519	0.4832
0.3616	12.0	38748	0.5111
0.3586	13.0	41977	0.5285
0.3508	14.0	45206	0.5084
0.3572	15.0	48435	0.5134
0.3497	16.0	51664	0.5092
0.3431	17.0	54893	0.5354
0.3362	18.0	58122	0.5221
0.3582	19.0	61351	0.5250
0.3442	20.0	64580	0.5362
0.3299	21.0	67809	0.5433
0.3148	22.0	71038	0.5425
0.347	23.0	74267	0.5520
0.3233	24.0	77496	0.5601
0.3163	25.0	80725	0.5510

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1