--- library_name: transformers license: cc-by-nc-sa-4.0 base_model: microsoft/layoutlmv3-base tags: - generated_from_trainer datasets: - funsd model-index: - name: layoutlm-document results: [] --- # layoutlm-document This model is a fine-tuned version of [microsoft/layoutlmv3-base](https://huggingface.co/microsoft/layoutlmv3-base) on the funsd dataset. It achieves the following results on the evaluation set: - Loss: 0.7561 - Answer: {'precision': 0.7122692725298588, 'recall': 0.802937576499388, 'f1': 0.7548906789413118, 'number': 817} - Header: {'precision': 0.42063492063492064, 'recall': 0.44537815126050423, 'f1': 0.4326530612244898, 'number': 119} - Question: {'precision': 0.7776769509981851, 'recall': 0.7957288765088208, 'f1': 0.7865993575034419, 'number': 1077} - Overall Precision: 0.7287 - Overall Recall: 0.7779 - Overall F1: 0.7525 - Overall Accuracy: 0.7896 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 16 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 15 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| | 1.7564 | 1.0 | 10 | 1.4998 | {'precision': 0.08313725490196078, 'recall': 0.12974296205630356, 'f1': 0.10133843212237093, 'number': 817} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.1588785046728972, 'recall': 0.1894150417827298, 'f1': 0.17280813214739515, 'number': 1077} | 0.1211 | 0.1540 | 0.1356 | 0.4467 | | 1.3951 | 2.0 | 20 | 1.1811 | {'precision': 0.25396825396825395, 'recall': 0.37209302325581395, 'f1': 0.3018867924528302, 'number': 817} | {'precision': 0.15625, 'recall': 0.04201680672268908, 'f1': 0.06622516556291391, 'number': 119} | {'precision': 0.35285815102328866, 'recall': 0.46425255338904364, 'f1': 0.400962309542903, 'number': 1077} | 0.3057 | 0.4019 | 0.3473 | 0.5764 | | 1.13 | 3.0 | 30 | 0.9739 | {'precision': 0.3972746331236897, 'recall': 0.4638922888616891, 'f1': 0.4280067758328628, 'number': 817} | {'precision': 0.15789473684210525, 'recall': 0.07563025210084033, 'f1': 0.10227272727272725, 'number': 119} | {'precision': 0.42933333333333334, 'recall': 0.5979572887650882, 'f1': 0.4998059759410168, 'number': 1077} | 0.4110 | 0.5127 | 0.4562 | 0.6753 | | 0.9412 | 4.0 | 40 | 0.8708 | {'precision': 0.4682203389830508, 'recall': 0.5410036719706243, 'f1': 0.5019875070982396, 'number': 817} | {'precision': 0.21782178217821782, 'recall': 0.18487394957983194, 'f1': 0.2, 'number': 119} | {'precision': 0.5829187396351575, 'recall': 0.6527390900649953, 'f1': 0.615856329391152, 'number': 1077} | 0.5184 | 0.5797 | 0.5474 | 0.7122 | | 0.7809 | 5.0 | 50 | 0.7932 | {'precision': 0.6105990783410138, 'recall': 0.6487148102815178, 'f1': 0.629080118694362, 'number': 817} | {'precision': 0.24347826086956523, 'recall': 0.23529411764705882, 'f1': 0.23931623931623933, 'number': 119} | {'precision': 0.6091867469879518, 'recall': 0.7511606313834726, 'f1': 0.6727650727650728, 'number': 1077} | 0.5915 | 0.6791 | 0.6323 | 0.7425 | | 0.6602 | 6.0 | 60 | 0.7728 | {'precision': 0.5849639546858908, 'recall': 0.6952264381884945, 'f1': 0.6353467561521253, 'number': 817} | {'precision': 0.23595505617977527, 'recall': 0.17647058823529413, 'f1': 0.20192307692307693, 'number': 119} | {'precision': 0.6678352322524101, 'recall': 0.7075208913649025, 'f1': 0.6871055004508566, 'number': 1077} | 0.6138 | 0.6711 | 0.6412 | 0.7490 | | 0.564 | 7.0 | 70 | 0.7390 | {'precision': 0.6692563817980022, 'recall': 0.7380660954712362, 'f1': 0.7019790454016297, 'number': 817} | {'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119} | {'precision': 0.6548808608762491, 'recall': 0.7910863509749304, 'f1': 0.7165685449957948, 'number': 1077} | 0.6399 | 0.7397 | 0.6862 | 0.7689 | | 0.494 | 8.0 | 80 | 0.7376 | {'precision': 0.7046750285062714, 'recall': 0.7564259485924113, 'f1': 0.7296340023612751, 'number': 817} | {'precision': 0.2824427480916031, 'recall': 0.31092436974789917, 'f1': 0.29600000000000004, 'number': 119} | {'precision': 0.6681957186544343, 'recall': 0.8115134633240483, 'f1': 0.7329140461215932, 'number': 1077} | 0.6602 | 0.7596 | 0.7064 | 0.7714 | | 0.451 | 9.0 | 90 | 0.7304 | {'precision': 0.7074756229685807, 'recall': 0.799265605875153, 'f1': 0.7505747126436781, 'number': 817} | {'precision': 0.3333333333333333, 'recall': 0.46218487394957986, 'f1': 0.3873239436619718, 'number': 119} | {'precision': 0.7134404057480981, 'recall': 0.7836583101207056, 'f1': 0.7469026548672568, 'number': 1077} | 0.6834 | 0.7710 | 0.7246 | 0.7738 | | 0.4322 | 10.0 | 100 | 0.7547 | {'precision': 0.7004310344827587, 'recall': 0.795593635250918, 'f1': 0.7449856733524355, 'number': 817} | {'precision': 0.2887323943661972, 'recall': 0.3445378151260504, 'f1': 0.31417624521072796, 'number': 119} | {'precision': 0.7091503267973857, 'recall': 0.8059424326833797, 'f1': 0.7544545849630595, 'number': 1077} | 0.6796 | 0.7745 | 0.7239 | 0.7750 | | 0.3682 | 11.0 | 110 | 0.7389 | {'precision': 0.7153518123667377, 'recall': 0.8212974296205631, 'f1': 0.7646723646723647, 'number': 817} | {'precision': 0.3219178082191781, 'recall': 0.3949579831932773, 'f1': 0.3547169811320755, 'number': 119} | {'precision': 0.7476635514018691, 'recall': 0.8170844939647168, 'f1': 0.7808340727595386, 'number': 1077} | 0.7068 | 0.7938 | 0.7478 | 0.7860 | | 0.3623 | 12.0 | 120 | 0.7472 | {'precision': 0.6998916576381365, 'recall': 0.7906976744186046, 'f1': 0.7425287356321839, 'number': 817} | {'precision': 0.373015873015873, 'recall': 0.3949579831932773, 'f1': 0.38367346938775515, 'number': 119} | {'precision': 0.7753818508535489, 'recall': 0.8012999071494893, 'f1': 0.7881278538812785, 'number': 1077} | 0.7197 | 0.7730 | 0.7454 | 0.7813 | | 0.3235 | 13.0 | 130 | 0.7432 | {'precision': 0.7095032397408207, 'recall': 0.8041615667074663, 'f1': 0.7538726333907055, 'number': 817} | {'precision': 0.38620689655172413, 'recall': 0.47058823529411764, 'f1': 0.42424242424242425, 'number': 119} | {'precision': 0.7731864095500459, 'recall': 0.7818012999071495, 'f1': 0.7774699907663898, 'number': 1077} | 0.7199 | 0.7725 | 0.7453 | 0.7850 | | 0.3052 | 14.0 | 140 | 0.7459 | {'precision': 0.7206040992448759, 'recall': 0.817625458996328, 'f1': 0.7660550458715596, 'number': 817} | {'precision': 0.4065040650406504, 'recall': 0.42016806722689076, 'f1': 0.4132231404958677, 'number': 119} | {'precision': 0.7706502636203867, 'recall': 0.8142989786443825, 'f1': 0.7918735891647856, 'number': 1077} | 0.7290 | 0.7923 | 0.7593 | 0.7897 | | 0.3046 | 15.0 | 150 | 0.7561 | {'precision': 0.7122692725298588, 'recall': 0.802937576499388, 'f1': 0.7548906789413118, 'number': 817} | {'precision': 0.42063492063492064, 'recall': 0.44537815126050423, 'f1': 0.4326530612244898, 'number': 119} | {'precision': 0.7776769509981851, 'recall': 0.7957288765088208, 'f1': 0.7865993575034419, 'number': 1077} | 0.7287 | 0.7779 | 0.7525 | 0.7896 | ### Framework versions - Transformers 4.44.2 - Pytorch 2.4.1+cu121 - Datasets 3.0.1 - Tokenizers 0.19.1