File size: 9,290 Bytes
bdea7b0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
---
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7037
- Answer: {'precision': 0.7206703910614525, 'recall': 0.7972805933250927, 'f1': 0.7570422535211268, 'number': 809}
- Header: {'precision': 0.3006993006993007, 'recall': 0.36134453781512604, 'f1': 0.3282442748091603, 'number': 119}
- Question: {'precision': 0.7585004359197908, 'recall': 0.8169014084507042, 'f1': 0.7866184448462928, 'number': 1065}
- Overall Precision: 0.7130
- Overall Recall: 0.7817
- Overall F1: 0.7458
- Overall Accuracy: 0.7989
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7815 | 1.0 | 10 | 1.5703 | {'precision': 0.022222222222222223, 'recall': 0.022249690976514216, 'f1': 0.022235948116121063, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.21046643913538113, 'recall': 0.17370892018779344, 'f1': 0.19032921810699588, 'number': 1065} | 0.1202 | 0.1019 | 0.1103 | 0.3789 |
| 1.4352 | 2.0 | 20 | 1.2331 | {'precision': 0.12166172106824925, 'recall': 0.10135970333745364, 'f1': 0.11058664868509778, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4863247863247863, 'recall': 0.5342723004694836, 'f1': 0.50917225950783, 'number': 1065} | 0.3530 | 0.3266 | 0.3393 | 0.5662 |
| 1.0804 | 3.0 | 30 | 0.9725 | {'precision': 0.4528985507246377, 'recall': 0.4635352286773795, 'f1': 0.4581551618814906, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6272806255430061, 'recall': 0.6779342723004694, 'f1': 0.6516245487364621, 'number': 1065} | 0.5447 | 0.5504 | 0.5475 | 0.6845 |
| 0.8495 | 4.0 | 40 | 0.7990 | {'precision': 0.5910973084886129, 'recall': 0.7058096415327565, 'f1': 0.6433802816901407, 'number': 809} | {'precision': 0.05970149253731343, 'recall': 0.03361344537815126, 'f1': 0.04301075268817204, 'number': 119} | {'precision': 0.6702222222222223, 'recall': 0.707981220657277, 'f1': 0.6885844748858447, 'number': 1065} | 0.6158 | 0.6668 | 0.6403 | 0.7510 |
| 0.6866 | 5.0 | 50 | 0.7357 | {'precision': 0.6541436464088398, 'recall': 0.7317676143386898, 'f1': 0.6907817969661612, 'number': 809} | {'precision': 0.2235294117647059, 'recall': 0.15966386554621848, 'f1': 0.18627450980392157, 'number': 119} | {'precision': 0.7028619528619529, 'recall': 0.784037558685446, 'f1': 0.7412339103417664, 'number': 1065} | 0.6639 | 0.7255 | 0.6934 | 0.7698 |
| 0.5626 | 6.0 | 60 | 0.6982 | {'precision': 0.6594871794871795, 'recall': 0.7948084054388134, 'f1': 0.7208520179372198, 'number': 809} | {'precision': 0.28378378378378377, 'recall': 0.17647058823529413, 'f1': 0.21761658031088082, 'number': 119} | {'precision': 0.6939417781274587, 'recall': 0.828169014084507, 'f1': 0.7551369863013697, 'number': 1065} | 0.6664 | 0.7757 | 0.7169 | 0.7872 |
| 0.4875 | 7.0 | 70 | 0.6710 | {'precision': 0.6905286343612335, 'recall': 0.7750309023485785, 'f1': 0.7303436225975539, 'number': 809} | {'precision': 0.2336448598130841, 'recall': 0.21008403361344538, 'f1': 0.22123893805309733, 'number': 119} | {'precision': 0.7287145242070117, 'recall': 0.819718309859155, 'f1': 0.7715422006186478, 'number': 1065} | 0.6891 | 0.7652 | 0.7252 | 0.7924 |
| 0.4499 | 8.0 | 80 | 0.6635 | {'precision': 0.6888412017167382, 'recall': 0.7935723114956736, 'f1': 0.7375071797817346, 'number': 809} | {'precision': 0.25210084033613445, 'recall': 0.25210084033613445, 'f1': 0.25210084033613445, 'number': 119} | {'precision': 0.7314814814814815, 'recall': 0.815962441314554, 'f1': 0.771415889924545, 'number': 1065} | 0.6883 | 0.7732 | 0.7283 | 0.7977 |
| 0.3939 | 9.0 | 90 | 0.6686 | {'precision': 0.709070796460177, 'recall': 0.792336217552534, 'f1': 0.7483946293053124, 'number': 809} | {'precision': 0.24817518248175183, 'recall': 0.2857142857142857, 'f1': 0.265625, 'number': 119} | {'precision': 0.7311557788944724, 'recall': 0.819718309859155, 'f1': 0.7729083665338645, 'number': 1065} | 0.6926 | 0.7767 | 0.7323 | 0.7970 |
| 0.3522 | 10.0 | 100 | 0.6728 | {'precision': 0.7094668117519043, 'recall': 0.8059332509270705, 'f1': 0.7546296296296295, 'number': 809} | {'precision': 0.3135593220338983, 'recall': 0.31092436974789917, 'f1': 0.31223628691983124, 'number': 119} | {'precision': 0.7573149741824441, 'recall': 0.8262910798122066, 'f1': 0.7903008531656939, 'number': 1065} | 0.7135 | 0.7873 | 0.7486 | 0.8034 |
| 0.3124 | 11.0 | 110 | 0.6859 | {'precision': 0.7041800643086816, 'recall': 0.8121137206427689, 'f1': 0.7543053960964409, 'number': 809} | {'precision': 0.3076923076923077, 'recall': 0.3025210084033613, 'f1': 0.30508474576271183, 'number': 119} | {'precision': 0.7731316725978647, 'recall': 0.815962441314554, 'f1': 0.793969849246231, 'number': 1065} | 0.7185 | 0.7837 | 0.7497 | 0.8006 |
| 0.306 | 12.0 | 120 | 0.6947 | {'precision': 0.720489977728285, 'recall': 0.799752781211372, 'f1': 0.7580550673696543, 'number': 809} | {'precision': 0.2773722627737226, 'recall': 0.31932773109243695, 'f1': 0.296875, 'number': 119} | {'precision': 0.7567332754126846, 'recall': 0.8178403755868544, 'f1': 0.7861010830324908, 'number': 1065} | 0.7118 | 0.7807 | 0.7447 | 0.7987 |
| 0.283 | 13.0 | 130 | 0.6948 | {'precision': 0.7201783723522854, 'recall': 0.7985166872682324, 'f1': 0.7573270808909731, 'number': 809} | {'precision': 0.30597014925373134, 'recall': 0.3445378151260504, 'f1': 0.3241106719367589, 'number': 119} | {'precision': 0.7585004359197908, 'recall': 0.8169014084507042, 'f1': 0.7866184448462928, 'number': 1065} | 0.7149 | 0.7812 | 0.7466 | 0.8000 |
| 0.2726 | 14.0 | 140 | 0.7002 | {'precision': 0.7119205298013245, 'recall': 0.7972805933250927, 'f1': 0.7521865889212828, 'number': 809} | {'precision': 0.3049645390070922, 'recall': 0.36134453781512604, 'f1': 0.3307692307692308, 'number': 119} | {'precision': 0.762532981530343, 'recall': 0.8140845070422535, 'f1': 0.787465940054496, 'number': 1065} | 0.7120 | 0.7802 | 0.7446 | 0.8001 |
| 0.264 | 15.0 | 150 | 0.7037 | {'precision': 0.7206703910614525, 'recall': 0.7972805933250927, 'f1': 0.7570422535211268, 'number': 809} | {'precision': 0.3006993006993007, 'recall': 0.36134453781512604, 'f1': 0.3282442748091603, 'number': 119} | {'precision': 0.7585004359197908, 'recall': 0.8169014084507042, 'f1': 0.7866184448462928, 'number': 1065} | 0.7130 | 0.7817 | 0.7458 | 0.7989 |
### Framework versions
- Transformers 4.26.0
- Pytorch 1.13.1+cu116
- Datasets 2.9.0
- Tokenizers 0.13.2
|