layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6675
• Answer: {'precision': 0.7104972375690608, 'recall': 0.7948084054388134, 'f1': 0.750291715285881, 'number': 809}
• Header: {'precision': 0.2892561983471074, 'recall': 0.29411764705882354, 'f1': 0.2916666666666667, 'number': 119}
• Question: {'precision': 0.7677642980935875, 'recall': 0.831924882629108, 'f1': 0.7985579089680036, 'number': 1065}
• Overall Precision: 0.7174
• Overall Recall: 0.7847
• Overall F1: 0.7496
• Overall Accuracy: 0.8194

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7901	1.0	10	1.6070	{'precision': 0.019525801952580194, 'recall': 0.0173053152039555, 'f1': 0.018348623853211007, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.2396486825595985, 'recall': 0.17934272300469484, 'f1': 0.20515574650912996, 'number': 1065}	0.1354	0.1029	0.1169	0.3392
1.4547	2.0	20	1.2498	{'precision': 0.21739130434782608, 'recall': 0.22249690976514216, 'f1': 0.21991447770311545, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.464573268921095, 'recall': 0.5417840375586854, 'f1': 0.5002167316861725, 'number': 1065}	0.3655	0.3798	0.3725	0.5784
1.0779	3.0	30	0.9620	{'precision': 0.46195652173913043, 'recall': 0.42027194066749074, 'f1': 0.4401294498381877, 'number': 809}	{'precision': 0.05405405405405406, 'recall': 0.01680672268907563, 'f1': 0.02564102564102564, 'number': 119}	{'precision': 0.631484794275492, 'recall': 0.6629107981220658, 'f1': 0.6468163078332569, 'number': 1065}	0.5542	0.5258	0.5396	0.6890
0.8184	4.0	40	0.7715	{'precision': 0.624868282402529, 'recall': 0.7330037082818294, 'f1': 0.6746302616609784, 'number': 809}	{'precision': 0.1875, 'recall': 0.10084033613445378, 'f1': 0.1311475409836066, 'number': 119}	{'precision': 0.6657963446475196, 'recall': 0.7183098591549296, 'f1': 0.6910569105691057, 'number': 1065}	0.6337	0.6874	0.6594	0.7596
0.6687	5.0	50	0.6994	{'precision': 0.6322778345250255, 'recall': 0.765142150803461, 'f1': 0.692393736017897, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.20168067226890757, 'f1': 0.23645320197044337, 'number': 119}	{'precision': 0.7097902097902098, 'recall': 0.7624413145539906, 'f1': 0.7351742870076958, 'number': 1065}	0.6593	0.7301	0.6929	0.7815
0.5553	6.0	60	0.6586	{'precision': 0.6430769230769231, 'recall': 0.7750309023485785, 'f1': 0.702914798206278, 'number': 809}	{'precision': 0.325, 'recall': 0.2184873949579832, 'f1': 0.26130653266331655, 'number': 119}	{'precision': 0.70863599677159, 'recall': 0.8244131455399061, 'f1': 0.7621527777777778, 'number': 1065}	0.6674	0.7682	0.7143	0.7961
0.4897	7.0	70	0.6659	{'precision': 0.6706521739130434, 'recall': 0.7626699629171817, 'f1': 0.7137073452862926, 'number': 809}	{'precision': 0.26605504587155965, 'recall': 0.24369747899159663, 'f1': 0.2543859649122807, 'number': 119}	{'precision': 0.7519788918205804, 'recall': 0.8028169014084507, 'f1': 0.7765667574931879, 'number': 1065}	0.6930	0.7531	0.7218	0.7944
0.4407	8.0	80	0.6417	{'precision': 0.6666666666666666, 'recall': 0.7688504326328801, 'f1': 0.7141216991963261, 'number': 809}	{'precision': 0.2692307692307692, 'recall': 0.23529411764705882, 'f1': 0.25112107623318386, 'number': 119}	{'precision': 0.7383966244725738, 'recall': 0.8215962441314554, 'f1': 0.7777777777777778, 'number': 1065}	0.6863	0.7652	0.7236	0.8050
0.3954	9.0	90	0.6419	{'precision': 0.6933333333333334, 'recall': 0.7713226205191595, 'f1': 0.7302516091281451, 'number': 809}	{'precision': 0.2698412698412698, 'recall': 0.2857142857142857, 'f1': 0.27755102040816326, 'number': 119}	{'precision': 0.7418273260687342, 'recall': 0.8309859154929577, 'f1': 0.7838795394154118, 'number': 1065}	0.6954	0.7742	0.7327	0.8089
0.3554	10.0	100	0.6524	{'precision': 0.6996625421822272, 'recall': 0.7688504326328801, 'f1': 0.7326266195524146, 'number': 809}	{'precision': 0.2578125, 'recall': 0.2773109243697479, 'f1': 0.26720647773279355, 'number': 119}	{'precision': 0.7448979591836735, 'recall': 0.8225352112676056, 'f1': 0.781793842034806, 'number': 1065}	0.6981	0.7682	0.7315	0.8105
0.3193	11.0	110	0.6687	{'precision': 0.6944444444444444, 'recall': 0.7725587144622992, 'f1': 0.7314218841427736, 'number': 809}	{'precision': 0.3076923076923077, 'recall': 0.2689075630252101, 'f1': 0.28699551569506726, 'number': 119}	{'precision': 0.7702349869451697, 'recall': 0.8309859154929577, 'f1': 0.7994579945799458, 'number': 1065}	0.7162	0.7737	0.7438	0.8105
0.3077	12.0	120	0.6657	{'precision': 0.7019650655021834, 'recall': 0.7948084054388134, 'f1': 0.7455072463768115, 'number': 809}	{'precision': 0.3125, 'recall': 0.29411764705882354, 'f1': 0.30303030303030304, 'number': 119}	{'precision': 0.7712532865907099, 'recall': 0.8262910798122066, 'f1': 0.7978241160471442, 'number': 1065}	0.7183	0.7817	0.7487	0.8127
0.2875	13.0	130	0.6820	{'precision': 0.6990950226244343, 'recall': 0.7639060568603214, 'f1': 0.7300649734199646, 'number': 809}	{'precision': 0.2608695652173913, 'recall': 0.3025210084033613, 'f1': 0.28015564202334625, 'number': 119}	{'precision': 0.7584415584415585, 'recall': 0.8225352112676056, 'f1': 0.7891891891891892, 'number': 1065}	0.7028	0.7677	0.7338	0.8094
0.2763	14.0	140	0.6680	{'precision': 0.7062706270627063, 'recall': 0.7935723114956736, 'f1': 0.7473806752037252, 'number': 809}	{'precision': 0.28688524590163933, 'recall': 0.29411764705882354, 'f1': 0.2904564315352697, 'number': 119}	{'precision': 0.7674216027874564, 'recall': 0.8272300469483568, 'f1': 0.7962042476276546, 'number': 1065}	0.7150	0.7817	0.7469	0.8181
0.2776	15.0	150	0.6675	{'precision': 0.7104972375690608, 'recall': 0.7948084054388134, 'f1': 0.750291715285881, 'number': 809}	{'precision': 0.2892561983471074, 'recall': 0.29411764705882354, 'f1': 0.2916666666666667, 'number': 119}	{'precision': 0.7677642980935875, 'recall': 0.831924882629108, 'f1': 0.7985579089680036, 'number': 1065}	0.7174	0.7847	0.7496	0.8194

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu116
• Datasets 2.10.1
• Tokenizers 0.13.2