lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5422
• Answer: {'precision': 0.8987804878048781, 'recall': 0.9020807833537332, 'f1': 0.9004276114844226, 'number': 817}
• Header: {'precision': 0.6528925619834711, 'recall': 0.6638655462184874, 'f1': 0.6583333333333333, 'number': 119}
• Question: {'precision': 0.8772845953002611, 'recall': 0.935933147632312, 'f1': 0.9056603773584906, 'number': 1077}
• Overall Precision: 0.8727
• Overall Recall: 0.9061
• Overall F1: 0.8891
• Overall Accuracy: 0.8153

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4117	10.5263	200	1.1040	{'precision': 0.8423475258918297, 'recall': 0.8959608323133414, 'f1': 0.8683274021352312, 'number': 817}	{'precision': 0.4925373134328358, 'recall': 0.5546218487394958, 'f1': 0.5217391304347825, 'number': 119}	{'precision': 0.8823529411764706, 'recall': 0.8913649025069638, 'f1': 0.8868360277136259, 'number': 1077}	0.8407	0.8733	0.8567	0.7772
0.0494	21.0526	400	1.4246	{'precision': 0.8377142857142857, 'recall': 0.8971848225214198, 'f1': 0.8664302600472813, 'number': 817}	{'precision': 0.5655737704918032, 'recall': 0.5798319327731093, 'f1': 0.5726141078838175, 'number': 119}	{'precision': 0.8783542039355993, 'recall': 0.9117920148560817, 'f1': 0.894760820045558, 'number': 1077}	0.8435	0.8862	0.8643	0.7895
0.0149	31.5789	600	1.3688	{'precision': 0.8272827282728272, 'recall': 0.9204406364749081, 'f1': 0.8713789107763616, 'number': 817}	{'precision': 0.6436781609195402, 'recall': 0.47058823529411764, 'f1': 0.5436893203883495, 'number': 119}	{'precision': 0.8790613718411552, 'recall': 0.904363974001857, 'f1': 0.891533180778032, 'number': 1077}	0.8470	0.8852	0.8657	0.8066
0.0074	42.1053	800	1.5512	{'precision': 0.8574739281575898, 'recall': 0.9057527539779682, 'f1': 0.880952380952381, 'number': 817}	{'precision': 0.6140350877192983, 'recall': 0.5882352941176471, 'f1': 0.6008583690987125, 'number': 119}	{'precision': 0.8713656387665198, 'recall': 0.9182915506035283, 'f1': 0.8942133815551537, 'number': 1077}	0.8518	0.8937	0.8722	0.7908
0.004	52.6316	1000	1.5808	{'precision': 0.8884892086330936, 'recall': 0.9069767441860465, 'f1': 0.8976377952755905, 'number': 817}	{'precision': 0.6239316239316239, 'recall': 0.6134453781512605, 'f1': 0.6186440677966102, 'number': 119}	{'precision': 0.8902991840435177, 'recall': 0.9117920148560817, 'f1': 0.9009174311926607, 'number': 1077}	0.8744	0.8922	0.8832	0.8017
0.0025	63.1579	1200	1.6746	{'precision': 0.8933333333333333, 'recall': 0.9020807833537332, 'f1': 0.8976857490864798, 'number': 817}	{'precision': 0.5873015873015873, 'recall': 0.6218487394957983, 'f1': 0.6040816326530611, 'number': 119}	{'precision': 0.8998194945848376, 'recall': 0.9257195914577531, 'f1': 0.9125858123569794, 'number': 1077}	0.8781	0.8982	0.8880	0.7978
0.0014	73.6842	1400	1.5175	{'precision': 0.8628841607565012, 'recall': 0.8935128518971848, 'f1': 0.8779314491882141, 'number': 817}	{'precision': 0.5833333333333334, 'recall': 0.5882352941176471, 'f1': 0.5857740585774059, 'number': 119}	{'precision': 0.8736933797909407, 'recall': 0.9312906220984215, 'f1': 0.9015730337078652, 'number': 1077}	0.8529	0.8957	0.8738	0.8073
0.0007	84.2105	1600	1.5422	{'precision': 0.8987804878048781, 'recall': 0.9020807833537332, 'f1': 0.9004276114844226, 'number': 817}	{'precision': 0.6528925619834711, 'recall': 0.6638655462184874, 'f1': 0.6583333333333333, 'number': 119}	{'precision': 0.8772845953002611, 'recall': 0.935933147632312, 'f1': 0.9056603773584906, 'number': 1077}	0.8727	0.9061	0.8891	0.8153
0.0007	94.7368	1800	1.6505	{'precision': 0.8702380952380953, 'recall': 0.8947368421052632, 'f1': 0.8823174411587205, 'number': 817}	{'precision': 0.5703125, 'recall': 0.6134453781512605, 'f1': 0.5910931174089069, 'number': 119}	{'precision': 0.8894927536231884, 'recall': 0.9117920148560817, 'f1': 0.9005043558000918, 'number': 1077}	0.8620	0.8872	0.8744	0.8034
0.0007	105.2632	2000	1.6541	{'precision': 0.8685446009389671, 'recall': 0.9057527539779682, 'f1': 0.8867585380467345, 'number': 817}	{'precision': 0.6095238095238096, 'recall': 0.5378151260504201, 'f1': 0.5714285714285715, 'number': 119}	{'precision': 0.8938700823421775, 'recall': 0.9071494893221913, 'f1': 0.9004608294930875, 'number': 1077}	0.8688	0.8847	0.8767	0.8024
0.0004	115.7895	2200	1.6276	{'precision': 0.866902237926973, 'recall': 0.9008567931456548, 'f1': 0.8835534213685473, 'number': 817}	{'precision': 0.64, 'recall': 0.5378151260504201, 'f1': 0.5844748858447488, 'number': 119}	{'precision': 0.8941704035874439, 'recall': 0.9257195914577531, 'f1': 0.9096715328467153, 'number': 1077}	0.8706	0.8927	0.8815	0.8082
0.0003	126.3158	2400	1.6682	{'precision': 0.8790419161676647, 'recall': 0.8984088127294981, 'f1': 0.8886198547215496, 'number': 817}	{'precision': 0.5964912280701754, 'recall': 0.5714285714285714, 'f1': 0.5836909871244635, 'number': 119}	{'precision': 0.8928892889288929, 'recall': 0.9210770659238626, 'f1': 0.9067641681901281, 'number': 1077}	0.8709	0.8912	0.8809	0.8042

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cpu
• Datasets 3.0.0
• Tokenizers 0.19.1