|
--- |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: layoutlm-funsd |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# layoutlm-funsd |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.8261 |
|
- Answer: {'precision': 0.5727482678983834, 'recall': 0.6131025957972805, 'f1': 0.5922388059701492, 'number': 809} |
|
- Header: {'precision': 0.09302325581395349, 'recall': 0.03361344537815126, 'f1': 0.04938271604938272, 'number': 119} |
|
- Question: {'precision': 0.6384228187919463, 'recall': 0.7145539906103286, 'f1': 0.6743464776251661, 'number': 1065} |
|
- Overall Precision: 0.6002 |
|
- Overall Recall: 0.6327 |
|
- Overall F1: 0.6160 |
|
- Overall Accuracy: 0.7523 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 64 |
|
- eval_batch_size: 32 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.9264 | 1.0 | 3 | 1.7763 | {'precision': 0.011029411764705883, 'recall': 0.022249690976514216, 'f1': 0.01474805407619828, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.09483960948396095, 'recall': 0.12769953051643193, 'f1': 0.108843537414966, 'number': 1065} | 0.0483 | 0.0773 | 0.0595 | 0.3277 | |
|
| 1.7361 | 2.0 | 6 | 1.6376 | {'precision': 0.0064754856614246065, 'recall': 0.00865265760197775, 'f1': 0.007407407407407408, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.17735470941883769, 'recall': 0.16619718309859155, 'f1': 0.17159476490547748, 'number': 1065} | 0.0885 | 0.0923 | 0.0904 | 0.3852 | |
|
| 1.6212 | 3.0 | 9 | 1.5225 | {'precision': 0.02002002002002002, 'recall': 0.024721878862793572, 'f1': 0.022123893805309734, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.27049180327868855, 'recall': 0.27887323943661974, 'f1': 0.27461858529819694, 'number': 1065} | 0.1512 | 0.1591 | 0.1550 | 0.4422 | |
|
| 1.5178 | 4.0 | 12 | 1.4133 | {'precision': 0.05408388520971302, 'recall': 0.06056860321384425, 'f1': 0.05714285714285715, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.313953488372093, 'recall': 0.38028169014084506, 'f1': 0.34394904458598724, 'number': 1065} | 0.2067 | 0.2278 | 0.2168 | 0.5062 | |
|
| 1.3853 | 5.0 | 15 | 1.3086 | {'precision': 0.08221024258760108, 'recall': 0.0754017305315204, 'f1': 0.07865892972275951, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.3780202650038971, 'recall': 0.45539906103286387, 'f1': 0.4131175468483816, 'number': 1065} | 0.2696 | 0.2740 | 0.2718 | 0.5453 | |
|
| 1.2546 | 6.0 | 18 | 1.2110 | {'precision': 0.1463768115942029, 'recall': 0.12484548825710753, 'f1': 0.13475650433622416, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4551282051282051, 'recall': 0.5333333333333333, 'f1': 0.49113705144833547, 'number': 1065} | 0.3452 | 0.3357 | 0.3404 | 0.5822 | |
|
| 1.1842 | 7.0 | 21 | 1.1217 | {'precision': 0.2563739376770538, 'recall': 0.22373300370828184, 'f1': 0.23894389438943894, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4908512330946698, 'recall': 0.5793427230046948, 'f1': 0.5314384151593453, 'number': 1065} | 0.4055 | 0.4004 | 0.4029 | 0.6223 | |
|
| 1.0564 | 8.0 | 24 | 1.0490 | {'precision': 0.364461738002594, 'recall': 0.3473423980222497, 'f1': 0.3556962025316456, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.509976057462091, 'recall': 0.6, 'f1': 0.551337359792925, 'number': 1065} | 0.4523 | 0.4616 | 0.4569 | 0.6679 | |
|
| 0.9865 | 9.0 | 27 | 0.9863 | {'precision': 0.4305555555555556, 'recall': 0.4215080346106304, 'f1': 0.42598376014990635, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5528726061615321, 'recall': 0.6234741784037559, 'f1': 0.5860547219770521, 'number': 1065} | 0.4978 | 0.5043 | 0.5010 | 0.6986 | |
|
| 0.9281 | 10.0 | 30 | 0.9357 | {'precision': 0.49454545454545457, 'recall': 0.5043263288009888, 'f1': 0.49938800489596086, 'number': 809} | {'precision': 0.034482758620689655, 'recall': 0.008403361344537815, 'f1': 0.013513513513513513, 'number': 119} | {'precision': 0.5873287671232876, 'recall': 0.644131455399061, 'f1': 0.6144200626959248, 'number': 1065} | 0.5415 | 0.5494 | 0.5455 | 0.7197 | |
|
| 0.8646 | 11.0 | 33 | 0.8968 | {'precision': 0.5333333333333333, 'recall': 0.5438813349814586, 'f1': 0.5385556915544676, 'number': 809} | {'precision': 0.0625, 'recall': 0.01680672268907563, 'f1': 0.026490066225165563, 'number': 119} | {'precision': 0.6031746031746031, 'recall': 0.6779342723004694, 'f1': 0.6383731211317418, 'number': 1065} | 0.5667 | 0.5840 | 0.5752 | 0.7344 | |
|
| 0.828 | 12.0 | 36 | 0.8653 | {'precision': 0.5617577197149644, 'recall': 0.584672435105068, 'f1': 0.5729860690490611, 'number': 809} | {'precision': 0.07692307692307693, 'recall': 0.025210084033613446, 'f1': 0.0379746835443038, 'number': 119} | {'precision': 0.6204013377926422, 'recall': 0.6967136150234742, 'f1': 0.6563467492260062, 'number': 1065} | 0.5864 | 0.6111 | 0.5985 | 0.7442 | |
|
| 0.7803 | 13.0 | 39 | 0.8442 | {'precision': 0.5667828106852497, 'recall': 0.6032138442521632, 'f1': 0.5844311377245508, 'number': 809} | {'precision': 0.07142857142857142, 'recall': 0.025210084033613446, 'f1': 0.037267080745341616, 'number': 119} | {'precision': 0.6343906510851419, 'recall': 0.7136150234741784, 'f1': 0.6716747680070703, 'number': 1065} | 0.5954 | 0.6277 | 0.6111 | 0.7504 | |
|
| 0.771 | 14.0 | 42 | 0.8312 | {'precision': 0.5679723502304147, 'recall': 0.6093943139678616, 'f1': 0.5879546809779368, 'number': 809} | {'precision': 0.09302325581395349, 'recall': 0.03361344537815126, 'f1': 0.04938271604938272, 'number': 119} | {'precision': 0.6376569037656904, 'recall': 0.7154929577464789, 'f1': 0.6743362831858407, 'number': 1065} | 0.5978 | 0.6317 | 0.6143 | 0.7516 | |
|
| 0.7843 | 15.0 | 45 | 0.8261 | {'precision': 0.5727482678983834, 'recall': 0.6131025957972805, 'f1': 0.5922388059701492, 'number': 809} | {'precision': 0.09302325581395349, 'recall': 0.03361344537815126, 'f1': 0.04938271604938272, 'number': 119} | {'precision': 0.6384228187919463, 'recall': 0.7145539906103286, 'f1': 0.6743464776251661, 'number': 1065} | 0.6002 | 0.6327 | 0.6160 | 0.7523 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.22.0.dev0 |
|
- Pytorch 1.12.1+cu116 |
|
- Datasets 2.4.0 |
|
- Tokenizers 0.12.1 |
|
|
