TenaliAI-FinTech-v1

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8406

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
2.3809	1.0	3852	1.9607
1.2279	2.0	7704	1.1452
0.9366	3.0	11556	0.9717
0.8443	4.0	15408	0.8946
0.795	5.0	19260	0.8747
0.736	6.0	23112	0.8618
0.7126	7.0	26964	0.8406
0.6937	8.0	30816	0.8497
0.6922	9.0	34668	0.8473
0.6521	10.0	38520	0.8563
0.6838	11.0	42372	0.8638
0.6395	12.0	46224	0.8615
0.6551	13.0	50076	0.8654
0.6038	14.0	53928	0.8795
0.6177	15.0	57780	0.8782
0.6164	16.0	61632	0.8914
0.6071	17.0	65484	0.8859
0.6125	18.0	69336	0.8921
0.6115	19.0	73188	0.8944
0.608	20.0	77040	0.9040
0.5943	21.0	80892	0.9133
0.632	22.0	84744	0.9031
0.6245	23.0	88596	0.9139
0.6094	24.0	92448	0.8965
0.6247	25.0	96300	0.9290

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1