contract_sections_with_labels_for_text_classification_v2

This model is a fine-tuned version of google-bert/bert-base-multilingual-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1120
• Accuracy: 0.9898

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.0995	1.0	1000	0.2133	0.9347
0.0474	2.0	2000	0.1669	0.97
0.0334	3.0	3000	0.1288	0.9778
0.0263	4.0	4000	0.1607	0.9782
0.0214	5.0	5000	0.1630	0.9778
0.0226	6.0	6000	0.1770	0.9712
0.0179	7.0	7000	0.1260	0.9835
0.0128	8.0	8000	0.1742	0.9755
0.0166	9.0	9000	0.1230	0.9835
0.0185	10.0	10000	0.1191	0.9848
0.0139	11.0	11000	0.1255	0.984
0.0147	12.0	12000	0.1247	0.9855
0.0086	13.0	13000	0.1242	0.9852
0.0062	14.0	14000	0.1209	0.9858
0.0067	15.0	15000	0.1205	0.9852
0.0087	16.0	16000	0.1295	0.9838
0.0086	17.0	17000	0.1319	0.9835
0.0047	18.0	18000	0.1165	0.9855
0.0076	19.0	19000	0.1166	0.983
0.0048	20.0	20000	0.1227	0.9855
0.0037	21.0	21000	0.1216	0.9855
0.0073	22.0	22000	0.1157	0.9848
0.0083	23.0	23000	0.1193	0.9832
0.0037	24.0	24000	0.1090	0.9862
0.01	25.0	25000	0.1183	0.9855
0.0063	26.0	26000	0.1129	0.9862
0.006	27.0	27000	0.1101	0.986
0.005	28.0	28000	0.1155	0.9838
0.0024	29.0	29000	0.1005	0.989
0.0067	30.0	30000	0.0998	0.9895
0.0054	31.0	31000	0.1235	0.9888
0.0064	32.0	32000	0.0966	0.9895
0.003	33.0	33000	0.1013	0.9902
0.0032	34.0	34000	0.1012	0.9898
0.0033	35.0	35000	0.0981	0.9895
0.0044	36.0	36000	0.1045	0.9898
0.0029	37.0	37000	0.1032	0.99
0.0043	38.0	38000	0.1370	0.9848
0.0055	39.0	39000	0.1015	0.9902
0.0031	40.0	40000	0.1029	0.9898
0.0023	41.0	41000	0.1013	0.9895
0.0051	42.0	42000	0.0969	0.9895
0.0035	43.0	43000	0.1028	0.9895
0.0055	44.0	44000	0.1237	0.985
0.0062	45.0	45000	0.1087	0.9895
0.0036	46.0	46000	0.1016	0.9908
0.0039	47.0	47000	0.1023	0.9908
0.0036	48.0	48000	0.1039	0.9902
0.003	49.0	49000	0.1290	0.9852
0.0029	50.0	50000	0.1092	0.9905
0.0054	51.0	51000	0.1053	0.9895
0.002	52.0	52000	0.1039	0.9898
0.0056	53.0	53000	0.1050	0.9892
0.0032	54.0	54000	0.0996	0.99
0.0042	55.0	55000	0.1013	0.9895
0.0033	56.0	56000	0.0965	0.9902
0.0035	57.0	57000	0.1027	0.9898
0.0049	58.0	58000	0.1016	0.9898
0.003	59.0	59000	0.0992	0.9902
0.0033	60.0	60000	0.1005	0.9902
0.0035	61.0	61000	0.1045	0.99
0.004	62.0	62000	0.1030	0.9902
0.0038	63.0	63000	0.1082	0.9905
0.0013	64.0	64000	0.1146	0.9895
0.0046	65.0	65000	0.1075	0.9905
0.0028	66.0	66000	0.1058	0.99
0.0064	67.0	67000	0.1019	0.9898
0.0035	68.0	68000	0.1061	0.9895
0.0036	69.0	69000	0.1086	0.9895
0.0014	70.0	70000	0.1112	0.9895
0.0031	71.0	71000	0.1104	0.9902
0.0022	72.0	72000	0.1099	0.9902
0.0041	73.0	73000	0.1068	0.99
0.0049	74.0	74000	0.1088	0.9898
0.0034	75.0	75000	0.1100	0.9898
0.0044	76.0	76000	0.1111	0.9898
0.0027	77.0	77000	0.1096	0.9898
0.0016	78.0	78000	0.1096	0.9898
0.0029	79.0	79000	0.1023	0.99
0.005	80.0	80000	0.1040	0.9898
0.0034	81.0	81000	0.1052	0.9898
0.0027	82.0	82000	0.1059	0.9898
0.0032	83.0	83000	0.1071	0.9898
0.0022	84.0	84000	0.1056	0.9898
0.0029	85.0	85000	0.1062	0.9898
0.0028	86.0	86000	0.1055	0.9898
0.0036	87.0	87000	0.1083	0.9898
0.0038	88.0	88000	0.1087	0.9898
0.0021	89.0	89000	0.1114	0.9898
0.0035	90.0	90000	0.1114	0.9898
0.0033	91.0	91000	0.1112	0.9898
0.0053	92.0	92000	0.1115	0.9898
0.0034	93.0	93000	0.1117	0.9898
0.0029	94.0	94000	0.1106	0.9898
0.0038	95.0	95000	0.1104	0.9898
0.0032	96.0	96000	0.1108	0.9898
0.0042	97.0	97000	0.1112	0.9898
0.0028	98.0	98000	0.1120	0.9898
0.0044	99.0	99000	0.1120	0.9898
0.0033	100.0	100000	0.1120	0.9898

Framework versions

• Transformers 4.42.4
• Pytorch 2.4.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1