layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7207
• Answer: {'precision': 0.7114754098360656, 'recall': 0.8046971569839307, 'f1': 0.7552204176334106, 'number': 809}
• Header: {'precision': 0.3025210084033613, 'recall': 0.3025210084033613, 'f1': 0.3025210084033613, 'number': 119}
• Question: {'precision': 0.7707061900610288, 'recall': 0.8300469483568075, 'f1': 0.7992766726943942, 'number': 1065}
• Overall Precision: 0.7203
• Overall Recall: 0.7883
• Overall F1: 0.7528
• Overall Accuracy: 0.7971

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8417	1.0	10	1.6166	{'precision': 0.028741328047571853, 'recall': 0.03584672435105068, 'f1': 0.0319031903190319, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.16827852998065765, 'recall': 0.16338028169014085, 'f1': 0.16579323487374942, 'number': 1065}	0.0993	0.1019	0.1006	0.3810
1.4476	2.0	20	1.2599	{'precision': 0.15711947626841244, 'recall': 0.11866501854140915, 'f1': 0.13521126760563382, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.46172441579371476, 'recall': 0.5380281690140845, 'f1': 0.4969644405897658, 'number': 1065}	0.3612	0.3357	0.3480	0.5747
1.1227	3.0	30	0.9780	{'precision': 0.4661214953271028, 'recall': 0.4932014833127318, 'f1': 0.4792792792792793, 'number': 809}	{'precision': 0.16279069767441862, 'recall': 0.058823529411764705, 'f1': 0.08641975308641975, 'number': 119}	{'precision': 0.5979020979020979, 'recall': 0.6422535211267606, 'f1': 0.6192847442281576, 'number': 1065}	0.5335	0.5469	0.5401	0.7036
0.8596	4.0	40	0.7946	{'precision': 0.5789473684210527, 'recall': 0.6526576019777504, 'f1': 0.6135967460778617, 'number': 809}	{'precision': 0.24193548387096775, 'recall': 0.12605042016806722, 'f1': 0.16574585635359115, 'number': 119}	{'precision': 0.6658141517476556, 'recall': 0.7333333333333333, 'f1': 0.6979445933869526, 'number': 1065}	0.6167	0.6643	0.6396	0.7589
0.6705	5.0	50	0.7132	{'precision': 0.6424759871931697, 'recall': 0.7441285537700866, 'f1': 0.6895761741122567, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.21008403361344538, 'f1': 0.2577319587628866, 'number': 119}	{'precision': 0.6747376916868443, 'recall': 0.7849765258215963, 'f1': 0.7256944444444444, 'number': 1065}	0.6499	0.7341	0.6894	0.7767
0.5653	6.0	60	0.6840	{'precision': 0.653125, 'recall': 0.7750309023485785, 'f1': 0.7088750706613907, 'number': 809}	{'precision': 0.30952380952380953, 'recall': 0.2184873949579832, 'f1': 0.2561576354679803, 'number': 119}	{'precision': 0.7077814569536424, 'recall': 0.8028169014084507, 'f1': 0.7523097228332599, 'number': 1065}	0.6696	0.7566	0.7105	0.7846
0.4959	7.0	70	0.6684	{'precision': 0.6872964169381107, 'recall': 0.7824474660074165, 'f1': 0.7317919075144509, 'number': 809}	{'precision': 0.2815533980582524, 'recall': 0.24369747899159663, 'f1': 0.26126126126126126, 'number': 119}	{'precision': 0.734006734006734, 'recall': 0.8187793427230047, 'f1': 0.7740790057700843, 'number': 1065}	0.6935	0.7697	0.7296	0.7950
0.4343	8.0	80	0.6696	{'precision': 0.6898395721925134, 'recall': 0.7972805933250927, 'f1': 0.7396788990825688, 'number': 809}	{'precision': 0.26956521739130435, 'recall': 0.2605042016806723, 'f1': 0.264957264957265, 'number': 119}	{'precision': 0.7495769881556683, 'recall': 0.831924882629108, 'f1': 0.7886070315976857, 'number': 1065}	0.6998	0.7837	0.7394	0.7987
0.375	9.0	90	0.6760	{'precision': 0.7105549510337323, 'recall': 0.8071693448702101, 'f1': 0.755787037037037, 'number': 809}	{'precision': 0.25, 'recall': 0.2605042016806723, 'f1': 0.25514403292181076, 'number': 119}	{'precision': 0.7758007117437722, 'recall': 0.8187793427230047, 'f1': 0.7967108268615805, 'number': 1065}	0.7180	0.7807	0.7481	0.7992
0.3532	10.0	100	0.6802	{'precision': 0.7147470398277718, 'recall': 0.8207663782447466, 'f1': 0.7640966628308401, 'number': 809}	{'precision': 0.3018867924528302, 'recall': 0.2689075630252101, 'f1': 0.28444444444444444, 'number': 119}	{'precision': 0.7761061946902655, 'recall': 0.8234741784037559, 'f1': 0.7990888382687927, 'number': 1065}	0.7266	0.7893	0.7566	0.8046
0.3265	11.0	110	0.6995	{'precision': 0.6968716289104638, 'recall': 0.7985166872682324, 'f1': 0.7442396313364056, 'number': 809}	{'precision': 0.308411214953271, 'recall': 0.2773109243697479, 'f1': 0.29203539823008845, 'number': 119}	{'precision': 0.7589134125636672, 'recall': 0.8394366197183099, 'f1': 0.7971466785555059, 'number': 1065}	0.7111	0.7893	0.7482	0.7973
0.3023	12.0	120	0.7053	{'precision': 0.7027896995708155, 'recall': 0.8096415327564895, 'f1': 0.7524411257897761, 'number': 809}	{'precision': 0.30303030303030304, 'recall': 0.33613445378151263, 'f1': 0.3187250996015936, 'number': 119}	{'precision': 0.769434628975265, 'recall': 0.8178403755868544, 'f1': 0.7928994082840236, 'number': 1065}	0.7131	0.7858	0.7477	0.7991
0.2927	13.0	130	0.7080	{'precision': 0.7024704618689581, 'recall': 0.8084054388133498, 'f1': 0.7517241379310345, 'number': 809}	{'precision': 0.3125, 'recall': 0.29411764705882354, 'f1': 0.30303030303030304, 'number': 119}	{'precision': 0.7679033649698016, 'recall': 0.8356807511737089, 'f1': 0.8003597122302158, 'number': 1065}	0.7171	0.7923	0.7528	0.7999
0.2756	14.0	140	0.7128	{'precision': 0.7081967213114754, 'recall': 0.8009888751545118, 'f1': 0.7517401392111368, 'number': 809}	{'precision': 0.32142857142857145, 'recall': 0.3025210084033613, 'f1': 0.3116883116883117, 'number': 119}	{'precision': 0.7720524017467248, 'recall': 0.8300469483568075, 'f1': 0.7999999999999999, 'number': 1065}	0.7219	0.7868	0.7529	0.7984
0.2741	15.0	150	0.7207	{'precision': 0.7114754098360656, 'recall': 0.8046971569839307, 'f1': 0.7552204176334106, 'number': 809}	{'precision': 0.3025210084033613, 'recall': 0.3025210084033613, 'f1': 0.3025210084033613, 'number': 119}	{'precision': 0.7707061900610288, 'recall': 0.8300469483568075, 'f1': 0.7992766726943942, 'number': 1065}	0.7203	0.7883	0.7528	0.7971

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.0
• Tokenizers 0.15.0