--- tags: - generated_from_trainer datasets: - funsd model-index: - name: layoutlm-funsd results: [] --- # layoutlm-funsd This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. It achieves the following results on the evaluation set: - Loss: 0.6924 - Answer: {'precision': 0.6991525423728814, 'recall': 0.8158220024721878, 'f1': 0.7529948659440959, 'number': 809} - Header: {'precision': 0.3253968253968254, 'recall': 0.3445378151260504, 'f1': 0.33469387755102037, 'number': 119} - Question: {'precision': 0.7807250221043325, 'recall': 0.8291079812206573, 'f1': 0.8041894353369764, 'number': 1065} - Overall Precision: 0.7197 - Overall Recall: 0.7948 - Overall F1: 0.7554 - Overall Accuracy: 0.8040 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 16 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 15 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| | 1.8027 | 1.0 | 10 | 1.6152 | {'precision': 0.0103359173126615, 'recall': 0.004944375772558714, 'f1': 0.006688963210702341, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.18232044198895028, 'recall': 0.061971830985915494, 'f1': 0.09250175192711983, 'number': 1065} | 0.0935 | 0.0351 | 0.0511 | 0.3249 | | 1.4899 | 2.0 | 20 | 1.2751 | {'precision': 0.18495297805642633, 'recall': 0.21878862793572312, 'f1': 0.20045300113250283, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4306969459671104, 'recall': 0.5164319248826291, 'f1': 0.46968403074295473, 'number': 1065} | 0.3254 | 0.3648 | 0.3440 | 0.5899 | | 1.1133 | 3.0 | 30 | 0.9514 | {'precision': 0.4911937377690802, 'recall': 0.6205191594561187, 'f1': 0.5483342435827417, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5651144435674822, 'recall': 0.672300469483568, 'f1': 0.614065180102916, 'number': 1065} | 0.5314 | 0.6111 | 0.5685 | 0.6982 | | 0.8513 | 4.0 | 40 | 0.8326 | {'precision': 0.5850746268656717, 'recall': 0.7268232385661311, 'f1': 0.6482910694597575, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6844003606853021, 'recall': 0.7126760563380282, 'f1': 0.6982520699172033, 'number': 1065} | 0.6248 | 0.6759 | 0.6493 | 0.7384 | | 0.7153 | 5.0 | 50 | 0.7422 | {'precision': 0.6159274193548387, 'recall': 0.7552533992583437, 'f1': 0.6785119378123265, 'number': 809} | {'precision': 0.08695652173913043, 'recall': 0.05042016806722689, 'f1': 0.06382978723404256, 'number': 119} | {'precision': 0.6826051112943117, 'recall': 0.7774647887323943, 'f1': 0.726953467954346, 'number': 1065} | 0.6354 | 0.7250 | 0.6773 | 0.7734 | | 0.5972 | 6.0 | 60 | 0.7031 | {'precision': 0.6330645161290323, 'recall': 0.7762669962917181, 'f1': 0.6973903387007219, 'number': 809} | {'precision': 0.15492957746478872, 'recall': 0.09243697478991597, 'f1': 0.11578947368421053, 'number': 119} | {'precision': 0.6802189210320563, 'recall': 0.8169014084507042, 'f1': 0.742320819112628, 'number': 1065} | 0.6443 | 0.7572 | 0.6962 | 0.7836 | | 0.5209 | 7.0 | 70 | 0.6902 | {'precision': 0.6597510373443983, 'recall': 0.7861557478368356, 'f1': 0.7174280879864636, 'number': 809} | {'precision': 0.2755102040816326, 'recall': 0.226890756302521, 'f1': 0.2488479262672811, 'number': 119} | {'precision': 0.7128463476070529, 'recall': 0.7971830985915493, 'f1': 0.7526595744680851, 'number': 1065} | 0.6711 | 0.7587 | 0.7122 | 0.7902 | | 0.4673 | 8.0 | 80 | 0.6693 | {'precision': 0.6649642492339122, 'recall': 0.8046971569839307, 'f1': 0.7281879194630874, 'number': 809} | {'precision': 0.28, 'recall': 0.23529411764705882, 'f1': 0.2557077625570776, 'number': 119} | {'precision': 0.7322314049586777, 'recall': 0.831924882629108, 'f1': 0.7789010989010988, 'number': 1065} | 0.6837 | 0.7852 | 0.7310 | 0.7965 | | 0.4151 | 9.0 | 90 | 0.6684 | {'precision': 0.6839323467230444, 'recall': 0.799752781211372, 'f1': 0.7373219373219373, 'number': 809} | {'precision': 0.30327868852459017, 'recall': 0.31092436974789917, 'f1': 0.3070539419087137, 'number': 119} | {'precision': 0.7363560033585222, 'recall': 0.8234741784037559, 'f1': 0.7774822695035462, 'number': 1065} | 0.6910 | 0.7832 | 0.7342 | 0.8017 | | 0.3689 | 10.0 | 100 | 0.6742 | {'precision': 0.6954643628509719, 'recall': 0.796044499381953, 'f1': 0.7423631123919308, 'number': 809} | {'precision': 0.3157894736842105, 'recall': 0.3025210084033613, 'f1': 0.30901287553648066, 'number': 119} | {'precision': 0.7483221476510067, 'recall': 0.8375586854460094, 'f1': 0.7904297740363314, 'number': 1065} | 0.7043 | 0.7888 | 0.7441 | 0.8000 | | 0.3327 | 11.0 | 110 | 0.6861 | {'precision': 0.6843198338525441, 'recall': 0.8145859085290482, 'f1': 0.7437923250564334, 'number': 809} | {'precision': 0.32456140350877194, 'recall': 0.31092436974789917, 'f1': 0.31759656652360513, 'number': 119} | {'precision': 0.7709790209790209, 'recall': 0.828169014084507, 'f1': 0.7985513807152557, 'number': 1065} | 0.7105 | 0.7918 | 0.7489 | 0.8031 | | 0.3167 | 12.0 | 120 | 0.6912 | {'precision': 0.6896186440677966, 'recall': 0.8046971569839307, 'f1': 0.7427267541357673, 'number': 809} | {'precision': 0.3416666666666667, 'recall': 0.3445378151260504, 'f1': 0.34309623430962344, 'number': 119} | {'precision': 0.7727272727272727, 'recall': 0.8300469483568075, 'f1': 0.8003621548211861, 'number': 1065} | 0.7138 | 0.7908 | 0.7503 | 0.8013 | | 0.3012 | 13.0 | 130 | 0.6878 | {'precision': 0.7015086206896551, 'recall': 0.8046971569839307, 'f1': 0.7495682210708117, 'number': 809} | {'precision': 0.33064516129032256, 'recall': 0.3445378151260504, 'f1': 0.33744855967078186, 'number': 119} | {'precision': 0.7694974003466204, 'recall': 0.8338028169014085, 'f1': 0.8003605227579991, 'number': 1065} | 0.7162 | 0.7928 | 0.7526 | 0.8073 | | 0.2882 | 14.0 | 140 | 0.6930 | {'precision': 0.6997885835095138, 'recall': 0.8182941903584673, 'f1': 0.7544159544159544, 'number': 809} | {'precision': 0.33064516129032256, 'recall': 0.3445378151260504, 'f1': 0.33744855967078186, 'number': 119} | {'precision': 0.7793468667255075, 'recall': 0.8291079812206573, 'f1': 0.8034576888080072, 'number': 1065} | 0.7199 | 0.7958 | 0.7560 | 0.8024 | | 0.2811 | 15.0 | 150 | 0.6924 | {'precision': 0.6991525423728814, 'recall': 0.8158220024721878, 'f1': 0.7529948659440959, 'number': 809} | {'precision': 0.3253968253968254, 'recall': 0.3445378151260504, 'f1': 0.33469387755102037, 'number': 119} | {'precision': 0.7807250221043325, 'recall': 0.8291079812206573, 'f1': 0.8041894353369764, 'number': 1065} | 0.7197 | 0.7948 | 0.7554 | 0.8040 | ### Framework versions - Transformers 4.26.1 - Pytorch 1.13.1+cu116 - Datasets 2.9.1.dev0 - Tokenizers 0.13.2