layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6993
• Answer: {'precision': 0.7155172413793104, 'recall': 0.8207663782447466, 'f1': 0.7645365572826713, 'number': 809}
• Header: {'precision': 0.2781954887218045, 'recall': 0.31092436974789917, 'f1': 0.2936507936507936, 'number': 119}
• Question: {'precision': 0.783303730017762, 'recall': 0.828169014084507, 'f1': 0.8051118210862619, 'number': 1065}
• Overall Precision: 0.7238
• Overall Recall: 0.7943
• Overall F1: 0.7574
• Overall Accuracy: 0.8095

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7894	1.0	10	1.6149	{'precision': 0.029508196721311476, 'recall': 0.03337453646477132, 'f1': 0.031322505800464036, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.18211920529801323, 'recall': 0.15492957746478872, 'f1': 0.167427701674277, 'number': 1065}	0.1053	0.0963	0.1006	0.3666
1.4628	2.0	20	1.2718	{'precision': 0.21764705882352942, 'recall': 0.22867737948084055, 'f1': 0.2230259192284509, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4429190751445087, 'recall': 0.5755868544600939, 'f1': 0.5006124948958759, 'number': 1065}	0.3572	0.4004	0.3776	0.5813
1.1079	3.0	30	0.9869	{'precision': 0.42190889370932755, 'recall': 0.48084054388133496, 'f1': 0.4494511842865395, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6041666666666666, 'recall': 0.6807511737089202, 'f1': 0.640176600441501, 'number': 1065}	0.5215	0.5590	0.5396	0.6898
0.8376	4.0	40	0.8064	{'precision': 0.6006036217303823, 'recall': 0.7379480840543882, 'f1': 0.6622296173044925, 'number': 809}	{'precision': 0.04918032786885246, 'recall': 0.025210084033613446, 'f1': 0.03333333333333334, 'number': 119}	{'precision': 0.6531302876480541, 'recall': 0.7248826291079812, 'f1': 0.6871384067645749, 'number': 1065}	0.6133	0.6884	0.6487	0.7512
0.6793	5.0	50	0.7442	{'precision': 0.6339468302658486, 'recall': 0.7663782447466008, 'f1': 0.693900391717963, 'number': 809}	{'precision': 0.15306122448979592, 'recall': 0.12605042016806722, 'f1': 0.1382488479262673, 'number': 119}	{'precision': 0.7100802854594113, 'recall': 0.7474178403755869, 'f1': 0.7282708142726441, 'number': 1065}	0.6513	0.7180	0.6831	0.7720
0.5643	6.0	60	0.6937	{'precision': 0.6551373346897253, 'recall': 0.796044499381953, 'f1': 0.7187499999999999, 'number': 809}	{'precision': 0.24175824175824176, 'recall': 0.18487394957983194, 'f1': 0.20952380952380953, 'number': 119}	{'precision': 0.71, 'recall': 0.8, 'f1': 0.752317880794702, 'number': 1065}	0.6675	0.7617	0.7115	0.7895
0.4869	7.0	70	0.6780	{'precision': 0.676130389064143, 'recall': 0.7948084054388134, 'f1': 0.7306818181818182, 'number': 809}	{'precision': 0.2072072072072072, 'recall': 0.19327731092436976, 'f1': 0.2, 'number': 119}	{'precision': 0.7147568013190437, 'recall': 0.8140845070422535, 'f1': 0.7611940298507464, 'number': 1065}	0.6738	0.7692	0.7184	0.7962
0.439	8.0	80	0.6706	{'precision': 0.696068012752391, 'recall': 0.8096415327564895, 'f1': 0.7485714285714284, 'number': 809}	{'precision': 0.2184873949579832, 'recall': 0.2184873949579832, 'f1': 0.2184873949579832, 'number': 119}	{'precision': 0.7454858125537404, 'recall': 0.8140845070422535, 'f1': 0.7782764811490125, 'number': 1065}	0.6964	0.7767	0.7343	0.8022
0.3922	9.0	90	0.6689	{'precision': 0.707742639040349, 'recall': 0.8022249690976514, 'f1': 0.7520278099652375, 'number': 809}	{'precision': 0.21774193548387097, 'recall': 0.226890756302521, 'f1': 0.2222222222222222, 'number': 119}	{'precision': 0.7601380500431406, 'recall': 0.8272300469483568, 'f1': 0.7922661870503598, 'number': 1065}	0.7077	0.7812	0.7427	0.8038
0.3518	10.0	100	0.6692	{'precision': 0.7065677966101694, 'recall': 0.8244746600741656, 'f1': 0.7609811751283514, 'number': 809}	{'precision': 0.23529411764705882, 'recall': 0.23529411764705882, 'f1': 0.23529411764705882, 'number': 119}	{'precision': 0.7663469921534438, 'recall': 0.8253521126760563, 'f1': 0.7947558770343581, 'number': 1065}	0.7122	0.7898	0.7490	0.8092
0.3165	11.0	110	0.6863	{'precision': 0.714902807775378, 'recall': 0.8182941903584673, 'f1': 0.7631123919308358, 'number': 809}	{'precision': 0.2631578947368421, 'recall': 0.29411764705882354, 'f1': 0.27777777777777773, 'number': 119}	{'precision': 0.7724867724867724, 'recall': 0.8225352112676056, 'f1': 0.7967257844474761, 'number': 1065}	0.7173	0.7893	0.7516	0.8083
0.3043	12.0	120	0.6898	{'precision': 0.7173678532901834, 'recall': 0.8220024721878862, 'f1': 0.7661290322580644, 'number': 809}	{'precision': 0.27692307692307694, 'recall': 0.3025210084033613, 'f1': 0.2891566265060241, 'number': 119}	{'precision': 0.7812223206377326, 'recall': 0.828169014084507, 'f1': 0.8040109389243391, 'number': 1065}	0.7242	0.7943	0.7576	0.8084
0.2853	13.0	130	0.6935	{'precision': 0.7167755991285403, 'recall': 0.8133498145859085, 'f1': 0.7620150550086855, 'number': 809}	{'precision': 0.30303030303030304, 'recall': 0.33613445378151263, 'f1': 0.3187250996015936, 'number': 119}	{'precision': 0.7855251544571933, 'recall': 0.8356807511737089, 'f1': 0.8098271155595996, 'number': 1065}	0.7274	0.7968	0.7605	0.8109
0.2724	14.0	140	0.6985	{'precision': 0.7212581344902386, 'recall': 0.8220024721878862, 'f1': 0.7683419988445985, 'number': 809}	{'precision': 0.2900763358778626, 'recall': 0.31932773109243695, 'f1': 0.304, 'number': 119}	{'precision': 0.786096256684492, 'recall': 0.828169014084507, 'f1': 0.8065843621399177, 'number': 1065}	0.7287	0.7953	0.7606	0.8091
0.2741	15.0	150	0.6993	{'precision': 0.7155172413793104, 'recall': 0.8207663782447466, 'f1': 0.7645365572826713, 'number': 809}	{'precision': 0.2781954887218045, 'recall': 0.31092436974789917, 'f1': 0.2936507936507936, 'number': 119}	{'precision': 0.783303730017762, 'recall': 0.828169014084507, 'f1': 0.8051118210862619, 'number': 1065}	0.7238	0.7943	0.7574	0.8095

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3