metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.6573
- Answer: {'precision': 0.7060773480662983, 'recall': 0.7898640296662547, 'f1': 0.7456242707117853, 'number': 809}
- Header: {'precision': 0.3333333333333333, 'recall': 0.3697478991596639, 'f1': 0.350597609561753, 'number': 119}
- Question: {'precision': 0.7687661777394306, 'recall': 0.8366197183098592, 'f1': 0.8012589928057554, 'number': 1065}
- Overall Precision: 0.7168
- Overall Recall: 0.7898
- Overall F1: 0.7515
- Overall Accuracy: 0.8172
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.7999 | 1.0 | 10 | 1.5802 | {'precision': 0.008905852417302799, 'recall': 0.00865265760197775, 'f1': 0.00877742946708464, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.1717325227963526, 'recall': 0.10610328638497653, 'f1': 0.13116656993615786, 'number': 1065} | 0.0831 | 0.0602 | 0.0698 | 0.3604 |
| 1.4567 | 2.0 | 20 | 1.2493 | {'precision': 0.18839103869653767, 'recall': 0.22867737948084055, 'f1': 0.20658849804578447, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.45693950177935944, 'recall': 0.6028169014084507, 'f1': 0.5198380566801619, 'number': 1065} | 0.3465 | 0.4150 | 0.3776 | 0.5986 |
| 1.114 | 3.0 | 30 | 0.9406 | {'precision': 0.43853820598006643, 'recall': 0.4894932014833127, 'f1': 0.46261682242990654, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5861538461538461, 'recall': 0.7154929577464789, 'f1': 0.6443974630021141, 'number': 1065} | 0.5237 | 0.5810 | 0.5509 | 0.7001 |
| 0.8434 | 4.0 | 40 | 0.7906 | {'precision': 0.5922836287799792, 'recall': 0.7021013597033374, 'f1': 0.6425339366515838, 'number': 809} | {'precision': 0.1111111111111111, 'recall': 0.04201680672268908, 'f1': 0.06097560975609755, 'number': 119} | {'precision': 0.6526994359387591, 'recall': 0.7605633802816901, 'f1': 0.7025151777970512, 'number': 1065} | 0.6160 | 0.6939 | 0.6527 | 0.7541 |
| 0.6817 | 5.0 | 50 | 0.7106 | {'precision': 0.6502192982456141, 'recall': 0.7330037082818294, 'f1': 0.6891342242882045, 'number': 809} | {'precision': 0.25301204819277107, 'recall': 0.17647058823529413, 'f1': 0.20792079207920794, 'number': 119} | {'precision': 0.683921568627451, 'recall': 0.8187793427230047, 'f1': 0.7452991452991454, 'number': 1065} | 0.6546 | 0.7456 | 0.6972 | 0.7854 |
| 0.5737 | 6.0 | 60 | 0.6807 | {'precision': 0.6482617586912065, 'recall': 0.7836835599505563, 'f1': 0.7095691102406267, 'number': 809} | {'precision': 0.273972602739726, 'recall': 0.16806722689075632, 'f1': 0.20833333333333331, 'number': 119} | {'precision': 0.717206132879046, 'recall': 0.7906103286384977, 'f1': 0.7521214828048235, 'number': 1065} | 0.6724 | 0.7506 | 0.7093 | 0.7898 |
| 0.5058 | 7.0 | 70 | 0.6538 | {'precision': 0.6564102564102564, 'recall': 0.7911001236093943, 'f1': 0.7174887892376681, 'number': 809} | {'precision': 0.3048780487804878, 'recall': 0.21008403361344538, 'f1': 0.24875621890547264, 'number': 119} | {'precision': 0.7324894514767932, 'recall': 0.8150234741784037, 'f1': 0.7715555555555556, 'number': 1065} | 0.6838 | 0.7692 | 0.7240 | 0.7996 |
| 0.4425 | 8.0 | 80 | 0.6574 | {'precision': 0.6625766871165644, 'recall': 0.8009888751545118, 'f1': 0.7252378287632905, 'number': 809} | {'precision': 0.3055555555555556, 'recall': 0.2773109243697479, 'f1': 0.2907488986784141, 'number': 119} | {'precision': 0.7365771812080537, 'recall': 0.8244131455399061, 'f1': 0.7780239255649092, 'number': 1065} | 0.6844 | 0.7822 | 0.7300 | 0.7999 |
| 0.3932 | 9.0 | 90 | 0.6375 | {'precision': 0.6876971608832808, 'recall': 0.8084054388133498, 'f1': 0.7431818181818182, 'number': 809} | {'precision': 0.3645833333333333, 'recall': 0.29411764705882354, 'f1': 0.3255813953488372, 'number': 119} | {'precision': 0.752129471890971, 'recall': 0.8291079812206573, 'f1': 0.7887449754354622, 'number': 1065} | 0.7078 | 0.7888 | 0.7461 | 0.8087 |
| 0.3798 | 10.0 | 100 | 0.6437 | {'precision': 0.6981541802388708, 'recall': 0.7948084054388134, 'f1': 0.7433526011560695, 'number': 809} | {'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119} | {'precision': 0.7665505226480837, 'recall': 0.8262910798122066, 'f1': 0.7953004970628107, 'number': 1065} | 0.7136 | 0.7837 | 0.7470 | 0.8098 |
| 0.3225 | 11.0 | 110 | 0.6566 | {'precision': 0.6817226890756303, 'recall': 0.8022249690976514, 'f1': 0.7370812038614423, 'number': 809} | {'precision': 0.336, 'recall': 0.35294117647058826, 'f1': 0.3442622950819672, 'number': 119} | {'precision': 0.7593856655290102, 'recall': 0.8356807511737089, 'f1': 0.7957085382208315, 'number': 1065} | 0.7030 | 0.7933 | 0.7454 | 0.8038 |
| 0.3097 | 12.0 | 120 | 0.6421 | {'precision': 0.6957928802588996, 'recall': 0.7972805933250927, 'f1': 0.7430875576036866, 'number': 809} | {'precision': 0.35, 'recall': 0.35294117647058826, 'f1': 0.35146443514644354, 'number': 119} | {'precision': 0.7692307692307693, 'recall': 0.8356807511737089, 'f1': 0.8010801080108011, 'number': 1065} | 0.7155 | 0.7913 | 0.7515 | 0.8177 |
| 0.2916 | 13.0 | 130 | 0.6515 | {'precision': 0.7035010940919038, 'recall': 0.7948084054388134, 'f1': 0.7463726059199072, 'number': 809} | {'precision': 0.33076923076923076, 'recall': 0.36134453781512604, 'f1': 0.34538152610441764, 'number': 119} | {'precision': 0.7649092480553155, 'recall': 0.8309859154929577, 'f1': 0.7965796579657966, 'number': 1065} | 0.7138 | 0.7883 | 0.7492 | 0.8154 |
| 0.2707 | 14.0 | 140 | 0.6557 | {'precision': 0.7016393442622951, 'recall': 0.7935723114956736, 'f1': 0.7447795823665894, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.36134453781512604, 'f1': 0.34677419354838707, 'number': 119} | {'precision': 0.7688966116420504, 'recall': 0.8309859154929577, 'f1': 0.7987364620938627, 'number': 1065} | 0.7153 | 0.7878 | 0.7498 | 0.8146 |
| 0.2729 | 15.0 | 150 | 0.6573 | {'precision': 0.7060773480662983, 'recall': 0.7898640296662547, 'f1': 0.7456242707117853, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.3697478991596639, 'f1': 0.350597609561753, 'number': 119} | {'precision': 0.7687661777394306, 'recall': 0.8366197183098592, 'f1': 0.8012589928057554, 'number': 1065} | 0.7168 | 0.7898 | 0.7515 | 0.8172 |
Framework versions
- Transformers 4.40.1
- Pytorch 2.3.0+cpu
- Datasets 2.19.0
- Tokenizers 0.19.1