layoutlm-funsd / README.md
riasharma's picture
End of training
5d1ba10
metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
  - generated_from_trainer
model-index:
  - name: layoutlm-funsd
    results: []

layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7207
  • Answer: {'precision': 0.7114754098360656, 'recall': 0.8046971569839307, 'f1': 0.7552204176334106, 'number': 809}
  • Header: {'precision': 0.3025210084033613, 'recall': 0.3025210084033613, 'f1': 0.3025210084033613, 'number': 119}
  • Question: {'precision': 0.7707061900610288, 'recall': 0.8300469483568075, 'f1': 0.7992766726943942, 'number': 1065}
  • Overall Precision: 0.7203
  • Overall Recall: 0.7883
  • Overall F1: 0.7528
  • Overall Accuracy: 0.7971

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 15
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Answer Header Question Overall Precision Overall Recall Overall F1 Overall Accuracy
1.8417 1.0 10 1.6166 {'precision': 0.028741328047571853, 'recall': 0.03584672435105068, 'f1': 0.0319031903190319, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.16827852998065765, 'recall': 0.16338028169014085, 'f1': 0.16579323487374942, 'number': 1065} 0.0993 0.1019 0.1006 0.3810
1.4476 2.0 20 1.2599 {'precision': 0.15711947626841244, 'recall': 0.11866501854140915, 'f1': 0.13521126760563382, 'number': 809} {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} {'precision': 0.46172441579371476, 'recall': 0.5380281690140845, 'f1': 0.4969644405897658, 'number': 1065} 0.3612 0.3357 0.3480 0.5747
1.1227 3.0 30 0.9780 {'precision': 0.4661214953271028, 'recall': 0.4932014833127318, 'f1': 0.4792792792792793, 'number': 809} {'precision': 0.16279069767441862, 'recall': 0.058823529411764705, 'f1': 0.08641975308641975, 'number': 119} {'precision': 0.5979020979020979, 'recall': 0.6422535211267606, 'f1': 0.6192847442281576, 'number': 1065} 0.5335 0.5469 0.5401 0.7036
0.8596 4.0 40 0.7946 {'precision': 0.5789473684210527, 'recall': 0.6526576019777504, 'f1': 0.6135967460778617, 'number': 809} {'precision': 0.24193548387096775, 'recall': 0.12605042016806722, 'f1': 0.16574585635359115, 'number': 119} {'precision': 0.6658141517476556, 'recall': 0.7333333333333333, 'f1': 0.6979445933869526, 'number': 1065} 0.6167 0.6643 0.6396 0.7589
0.6705 5.0 50 0.7132 {'precision': 0.6424759871931697, 'recall': 0.7441285537700866, 'f1': 0.6895761741122567, 'number': 809} {'precision': 0.3333333333333333, 'recall': 0.21008403361344538, 'f1': 0.2577319587628866, 'number': 119} {'precision': 0.6747376916868443, 'recall': 0.7849765258215963, 'f1': 0.7256944444444444, 'number': 1065} 0.6499 0.7341 0.6894 0.7767
0.5653 6.0 60 0.6840 {'precision': 0.653125, 'recall': 0.7750309023485785, 'f1': 0.7088750706613907, 'number': 809} {'precision': 0.30952380952380953, 'recall': 0.2184873949579832, 'f1': 0.2561576354679803, 'number': 119} {'precision': 0.7077814569536424, 'recall': 0.8028169014084507, 'f1': 0.7523097228332599, 'number': 1065} 0.6696 0.7566 0.7105 0.7846
0.4959 7.0 70 0.6684 {'precision': 0.6872964169381107, 'recall': 0.7824474660074165, 'f1': 0.7317919075144509, 'number': 809} {'precision': 0.2815533980582524, 'recall': 0.24369747899159663, 'f1': 0.26126126126126126, 'number': 119} {'precision': 0.734006734006734, 'recall': 0.8187793427230047, 'f1': 0.7740790057700843, 'number': 1065} 0.6935 0.7697 0.7296 0.7950
0.4343 8.0 80 0.6696 {'precision': 0.6898395721925134, 'recall': 0.7972805933250927, 'f1': 0.7396788990825688, 'number': 809} {'precision': 0.26956521739130435, 'recall': 0.2605042016806723, 'f1': 0.264957264957265, 'number': 119} {'precision': 0.7495769881556683, 'recall': 0.831924882629108, 'f1': 0.7886070315976857, 'number': 1065} 0.6998 0.7837 0.7394 0.7987
0.375 9.0 90 0.6760 {'precision': 0.7105549510337323, 'recall': 0.8071693448702101, 'f1': 0.755787037037037, 'number': 809} {'precision': 0.25, 'recall': 0.2605042016806723, 'f1': 0.25514403292181076, 'number': 119} {'precision': 0.7758007117437722, 'recall': 0.8187793427230047, 'f1': 0.7967108268615805, 'number': 1065} 0.7180 0.7807 0.7481 0.7992
0.3532 10.0 100 0.6802 {'precision': 0.7147470398277718, 'recall': 0.8207663782447466, 'f1': 0.7640966628308401, 'number': 809} {'precision': 0.3018867924528302, 'recall': 0.2689075630252101, 'f1': 0.28444444444444444, 'number': 119} {'precision': 0.7761061946902655, 'recall': 0.8234741784037559, 'f1': 0.7990888382687927, 'number': 1065} 0.7266 0.7893 0.7566 0.8046
0.3265 11.0 110 0.6995 {'precision': 0.6968716289104638, 'recall': 0.7985166872682324, 'f1': 0.7442396313364056, 'number': 809} {'precision': 0.308411214953271, 'recall': 0.2773109243697479, 'f1': 0.29203539823008845, 'number': 119} {'precision': 0.7589134125636672, 'recall': 0.8394366197183099, 'f1': 0.7971466785555059, 'number': 1065} 0.7111 0.7893 0.7482 0.7973
0.3023 12.0 120 0.7053 {'precision': 0.7027896995708155, 'recall': 0.8096415327564895, 'f1': 0.7524411257897761, 'number': 809} {'precision': 0.30303030303030304, 'recall': 0.33613445378151263, 'f1': 0.3187250996015936, 'number': 119} {'precision': 0.769434628975265, 'recall': 0.8178403755868544, 'f1': 0.7928994082840236, 'number': 1065} 0.7131 0.7858 0.7477 0.7991
0.2927 13.0 130 0.7080 {'precision': 0.7024704618689581, 'recall': 0.8084054388133498, 'f1': 0.7517241379310345, 'number': 809} {'precision': 0.3125, 'recall': 0.29411764705882354, 'f1': 0.30303030303030304, 'number': 119} {'precision': 0.7679033649698016, 'recall': 0.8356807511737089, 'f1': 0.8003597122302158, 'number': 1065} 0.7171 0.7923 0.7528 0.7999
0.2756 14.0 140 0.7128 {'precision': 0.7081967213114754, 'recall': 0.8009888751545118, 'f1': 0.7517401392111368, 'number': 809} {'precision': 0.32142857142857145, 'recall': 0.3025210084033613, 'f1': 0.3116883116883117, 'number': 119} {'precision': 0.7720524017467248, 'recall': 0.8300469483568075, 'f1': 0.7999999999999999, 'number': 1065} 0.7219 0.7868 0.7529 0.7984
0.2741 15.0 150 0.7207 {'precision': 0.7114754098360656, 'recall': 0.8046971569839307, 'f1': 0.7552204176334106, 'number': 809} {'precision': 0.3025210084033613, 'recall': 0.3025210084033613, 'f1': 0.3025210084033613, 'number': 119} {'precision': 0.7707061900610288, 'recall': 0.8300469483568075, 'f1': 0.7992766726943942, 'number': 1065} 0.7203 0.7883 0.7528 0.7971

Framework versions

  • Transformers 4.36.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.0
  • Tokenizers 0.15.0