lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7187
• Answer: {'precision': 0.8569767441860465, 'recall': 0.9020807833537332, 'f1': 0.8789505068574837, 'number': 817}
• Header: {'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119}
• Question: {'precision': 0.8962693357597816, 'recall': 0.914577530176416, 'f1': 0.9053308823529412, 'number': 1077}
• Overall Precision: 0.8671
• Overall Recall: 0.8882
• Overall F1: 0.8775
• Overall Accuracy: 0.7998

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4061	10.5263	200	1.0439	{'precision': 0.8062770562770563, 'recall': 0.9118727050183598, 'f1': 0.855829982768524, 'number': 817}	{'precision': 0.6043956043956044, 'recall': 0.46218487394957986, 'f1': 0.5238095238095237, 'number': 119}	{'precision': 0.8777173913043478, 'recall': 0.8997214484679665, 'f1': 0.8885832187070151, 'number': 1077}	0.8348	0.8788	0.8562	0.7914
0.0454	21.0526	400	1.4836	{'precision': 0.8243688254665203, 'recall': 0.9192166462668299, 'f1': 0.869212962962963, 'number': 817}	{'precision': 0.48872180451127817, 'recall': 0.5462184873949579, 'f1': 0.5158730158730158, 'number': 119}	{'precision': 0.9097963142580019, 'recall': 0.8709377901578459, 'f1': 0.889943074003795, 'number': 1077}	0.8453	0.8713	0.8581	0.7934
0.0145	31.5789	600	1.3593	{'precision': 0.8672248803827751, 'recall': 0.8873929008567931, 'f1': 0.8771929824561404, 'number': 817}	{'precision': 0.6018518518518519, 'recall': 0.5462184873949579, 'f1': 0.5726872246696034, 'number': 119}	{'precision': 0.8753339269813001, 'recall': 0.9127205199628597, 'f1': 0.8936363636363636, 'number': 1077}	0.8578	0.8808	0.8691	0.8041
0.0074	42.1053	800	1.5465	{'precision': 0.8311111111111111, 'recall': 0.9155446756425949, 'f1': 0.8712871287128713, 'number': 817}	{'precision': 0.6190476190476191, 'recall': 0.5462184873949579, 'f1': 0.5803571428571429, 'number': 119}	{'precision': 0.9031657355679702, 'recall': 0.9006499535747446, 'f1': 0.901906090190609, 'number': 1077}	0.8576	0.8857	0.8715	0.7981
0.0045	52.6316	1000	1.4018	{'precision': 0.8342922899884925, 'recall': 0.8873929008567931, 'f1': 0.860023724792408, 'number': 817}	{'precision': 0.5803571428571429, 'recall': 0.5462184873949579, 'f1': 0.5627705627705628, 'number': 119}	{'precision': 0.8855855855855855, 'recall': 0.9127205199628597, 'f1': 0.8989483310470964, 'number': 1077}	0.8479	0.8808	0.8640	0.8064
0.0031	63.1579	1200	1.6815	{'precision': 0.8714810281517748, 'recall': 0.8714810281517748, 'f1': 0.8714810281517748, 'number': 817}	{'precision': 0.6039603960396039, 'recall': 0.5126050420168067, 'f1': 0.5545454545454545, 'number': 119}	{'precision': 0.8730569948186528, 'recall': 0.9387186629526463, 'f1': 0.9046979865771813, 'number': 1077}	0.8593	0.8862	0.8726	0.7952
0.0018	73.6842	1400	1.5823	{'precision': 0.8553530751708428, 'recall': 0.9192166462668299, 'f1': 0.8861356932153391, 'number': 817}	{'precision': 0.5865384615384616, 'recall': 0.5126050420168067, 'f1': 0.5470852017937219, 'number': 119}	{'precision': 0.8930530164533821, 'recall': 0.9071494893221913, 'f1': 0.9000460617227084, 'number': 1077}	0.8618	0.8887	0.8750	0.8061
0.0015	84.2105	1600	1.6540	{'precision': 0.8509895227008148, 'recall': 0.8947368421052632, 'f1': 0.8723150357995225, 'number': 817}	{'precision': 0.6477272727272727, 'recall': 0.4789915966386555, 'f1': 0.5507246376811594, 'number': 119}	{'precision': 0.8776408450704225, 'recall': 0.9257195914577531, 'f1': 0.9010393131495708, 'number': 1077}	0.8569	0.8867	0.8716	0.8039
0.0005	94.7368	1800	1.7397	{'precision': 0.8578199052132701, 'recall': 0.8861689106487148, 'f1': 0.8717639975918122, 'number': 817}	{'precision': 0.5740740740740741, 'recall': 0.5210084033613446, 'f1': 0.5462555066079295, 'number': 119}	{'precision': 0.8785971223021583, 'recall': 0.9071494893221913, 'f1': 0.8926450433988122, 'number': 1077}	0.8542	0.8758	0.8649	0.7925
0.0003	105.2632	2000	1.6680	{'precision': 0.8688915375446961, 'recall': 0.8922888616891065, 'f1': 0.8804347826086957, 'number': 817}	{'precision': 0.6122448979591837, 'recall': 0.5042016806722689, 'f1': 0.5529953917050692, 'number': 119}	{'precision': 0.8774250440917107, 'recall': 0.9238625812441968, 'f1': 0.9000452284034374, 'number': 1077}	0.8614	0.8862	0.8737	0.8011
0.0002	115.7895	2200	1.6812	{'precision': 0.8494252873563218, 'recall': 0.9045287637698899, 'f1': 0.8761114404267932, 'number': 817}	{'precision': 0.6704545454545454, 'recall': 0.4957983193277311, 'f1': 0.5700483091787439, 'number': 119}	{'precision': 0.8914798206278027, 'recall': 0.9229340761374187, 'f1': 0.906934306569343, 'number': 1077}	0.8644	0.8902	0.8771	0.8051
0.0004	126.3158	2400	1.7187	{'precision': 0.8569767441860465, 'recall': 0.9020807833537332, 'f1': 0.8789505068574837, 'number': 817}	{'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119}	{'precision': 0.8962693357597816, 'recall': 0.914577530176416, 'f1': 0.9053308823529412, 'number': 1077}	0.8671	0.8882	0.8775	0.7998

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3