layoutlm-document

This model is a fine-tuned version of microsoft/layoutlmv3-base on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7561
• Answer: {'precision': 0.7122692725298588, 'recall': 0.802937576499388, 'f1': 0.7548906789413118, 'number': 817}
• Header: {'precision': 0.42063492063492064, 'recall': 0.44537815126050423, 'f1': 0.4326530612244898, 'number': 119}
• Question: {'precision': 0.7776769509981851, 'recall': 0.7957288765088208, 'f1': 0.7865993575034419, 'number': 1077}
• Overall Precision: 0.7287
• Overall Recall: 0.7779
• Overall F1: 0.7525
• Overall Accuracy: 0.7896

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7564	1.0	10	1.4998	{'precision': 0.08313725490196078, 'recall': 0.12974296205630356, 'f1': 0.10133843212237093, 'number': 817}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.1588785046728972, 'recall': 0.1894150417827298, 'f1': 0.17280813214739515, 'number': 1077}	0.1211	0.1540	0.1356	0.4467
1.3951	2.0	20	1.1811	{'precision': 0.25396825396825395, 'recall': 0.37209302325581395, 'f1': 0.3018867924528302, 'number': 817}	{'precision': 0.15625, 'recall': 0.04201680672268908, 'f1': 0.06622516556291391, 'number': 119}	{'precision': 0.35285815102328866, 'recall': 0.46425255338904364, 'f1': 0.400962309542903, 'number': 1077}	0.3057	0.4019	0.3473	0.5764
1.13	3.0	30	0.9739	{'precision': 0.3972746331236897, 'recall': 0.4638922888616891, 'f1': 0.4280067758328628, 'number': 817}	{'precision': 0.15789473684210525, 'recall': 0.07563025210084033, 'f1': 0.10227272727272725, 'number': 119}	{'precision': 0.42933333333333334, 'recall': 0.5979572887650882, 'f1': 0.4998059759410168, 'number': 1077}	0.4110	0.5127	0.4562	0.6753
0.9412	4.0	40	0.8708	{'precision': 0.4682203389830508, 'recall': 0.5410036719706243, 'f1': 0.5019875070982396, 'number': 817}	{'precision': 0.21782178217821782, 'recall': 0.18487394957983194, 'f1': 0.2, 'number': 119}	{'precision': 0.5829187396351575, 'recall': 0.6527390900649953, 'f1': 0.615856329391152, 'number': 1077}	0.5184	0.5797	0.5474	0.7122
0.7809	5.0	50	0.7932	{'precision': 0.6105990783410138, 'recall': 0.6487148102815178, 'f1': 0.629080118694362, 'number': 817}	{'precision': 0.24347826086956523, 'recall': 0.23529411764705882, 'f1': 0.23931623931623933, 'number': 119}	{'precision': 0.6091867469879518, 'recall': 0.7511606313834726, 'f1': 0.6727650727650728, 'number': 1077}	0.5915	0.6791	0.6323	0.7425
0.6602	6.0	60	0.7728	{'precision': 0.5849639546858908, 'recall': 0.6952264381884945, 'f1': 0.6353467561521253, 'number': 817}	{'precision': 0.23595505617977527, 'recall': 0.17647058823529413, 'f1': 0.20192307692307693, 'number': 119}	{'precision': 0.6678352322524101, 'recall': 0.7075208913649025, 'f1': 0.6871055004508566, 'number': 1077}	0.6138	0.6711	0.6412	0.7490
0.564	7.0	70	0.7390	{'precision': 0.6692563817980022, 'recall': 0.7380660954712362, 'f1': 0.7019790454016297, 'number': 817}	{'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119}	{'precision': 0.6548808608762491, 'recall': 0.7910863509749304, 'f1': 0.7165685449957948, 'number': 1077}	0.6399	0.7397	0.6862	0.7689
0.494	8.0	80	0.7376	{'precision': 0.7046750285062714, 'recall': 0.7564259485924113, 'f1': 0.7296340023612751, 'number': 817}	{'precision': 0.2824427480916031, 'recall': 0.31092436974789917, 'f1': 0.29600000000000004, 'number': 119}	{'precision': 0.6681957186544343, 'recall': 0.8115134633240483, 'f1': 0.7329140461215932, 'number': 1077}	0.6602	0.7596	0.7064	0.7714
0.451	9.0	90	0.7304	{'precision': 0.7074756229685807, 'recall': 0.799265605875153, 'f1': 0.7505747126436781, 'number': 817}	{'precision': 0.3333333333333333, 'recall': 0.46218487394957986, 'f1': 0.3873239436619718, 'number': 119}	{'precision': 0.7134404057480981, 'recall': 0.7836583101207056, 'f1': 0.7469026548672568, 'number': 1077}	0.6834	0.7710	0.7246	0.7738
0.4322	10.0	100	0.7547	{'precision': 0.7004310344827587, 'recall': 0.795593635250918, 'f1': 0.7449856733524355, 'number': 817}	{'precision': 0.2887323943661972, 'recall': 0.3445378151260504, 'f1': 0.31417624521072796, 'number': 119}	{'precision': 0.7091503267973857, 'recall': 0.8059424326833797, 'f1': 0.7544545849630595, 'number': 1077}	0.6796	0.7745	0.7239	0.7750
0.3682	11.0	110	0.7389	{'precision': 0.7153518123667377, 'recall': 0.8212974296205631, 'f1': 0.7646723646723647, 'number': 817}	{'precision': 0.3219178082191781, 'recall': 0.3949579831932773, 'f1': 0.3547169811320755, 'number': 119}	{'precision': 0.7476635514018691, 'recall': 0.8170844939647168, 'f1': 0.7808340727595386, 'number': 1077}	0.7068	0.7938	0.7478	0.7860
0.3623	12.0	120	0.7472	{'precision': 0.6998916576381365, 'recall': 0.7906976744186046, 'f1': 0.7425287356321839, 'number': 817}	{'precision': 0.373015873015873, 'recall': 0.3949579831932773, 'f1': 0.38367346938775515, 'number': 119}	{'precision': 0.7753818508535489, 'recall': 0.8012999071494893, 'f1': 0.7881278538812785, 'number': 1077}	0.7197	0.7730	0.7454	0.7813
0.3235	13.0	130	0.7432	{'precision': 0.7095032397408207, 'recall': 0.8041615667074663, 'f1': 0.7538726333907055, 'number': 817}	{'precision': 0.38620689655172413, 'recall': 0.47058823529411764, 'f1': 0.42424242424242425, 'number': 119}	{'precision': 0.7731864095500459, 'recall': 0.7818012999071495, 'f1': 0.7774699907663898, 'number': 1077}	0.7199	0.7725	0.7453	0.7850
0.3052	14.0	140	0.7459	{'precision': 0.7206040992448759, 'recall': 0.817625458996328, 'f1': 0.7660550458715596, 'number': 817}	{'precision': 0.4065040650406504, 'recall': 0.42016806722689076, 'f1': 0.4132231404958677, 'number': 119}	{'precision': 0.7706502636203867, 'recall': 0.8142989786443825, 'f1': 0.7918735891647856, 'number': 1077}	0.7290	0.7923	0.7593	0.7897
0.3046	15.0	150	0.7561	{'precision': 0.7122692725298588, 'recall': 0.802937576499388, 'f1': 0.7548906789413118, 'number': 817}	{'precision': 0.42063492063492064, 'recall': 0.44537815126050423, 'f1': 0.4326530612244898, 'number': 119}	{'precision': 0.7776769509981851, 'recall': 0.7957288765088208, 'f1': 0.7865993575034419, 'number': 1077}	0.7287	0.7779	0.7525	0.7896

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.19.1