layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6712
• Answer: {'precision': 0.6719409282700421, 'recall': 0.7873918417799752, 'f1': 0.7250996015936254, 'number': 809}
• Header: {'precision': 0.3153153153153153, 'recall': 0.29411764705882354, 'f1': 0.30434782608695654, 'number': 119}
• Question: {'precision': 0.7069109075770191, 'recall': 0.7971830985915493, 'f1': 0.7493380406001765, 'number': 1065}
• Overall Precision: 0.6730
• Overall Recall: 0.7632
• Overall F1: 0.7153
• Overall Accuracy: 0.7909

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7528	1.0	10	1.5450	{'precision': 0.04079497907949791, 'recall': 0.048207663782447466, 'f1': 0.04419263456090652, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.1806060606060606, 'recall': 0.13990610328638498, 'f1': 0.15767195767195766, 'number': 1065}	0.1056	0.0943	0.0996	0.3786
1.4294	2.0	20	1.2643	{'precision': 0.20842824601366744, 'recall': 0.22620519159456118, 'f1': 0.2169531713100178, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4424778761061947, 'recall': 0.5164319248826291, 'f1': 0.4766031195840555, 'number': 1065}	0.3456	0.3678	0.3563	0.5767
1.1277	3.0	30	0.9879	{'precision': 0.4243845252051583, 'recall': 0.44746600741656367, 'f1': 0.4356197352587245, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5726141078838174, 'recall': 0.647887323943662, 'f1': 0.6079295154185022, 'number': 1065}	0.5092	0.5278	0.5184	0.6932
0.8834	4.0	40	0.8188	{'precision': 0.574052812858783, 'recall': 0.6180469715698393, 'f1': 0.5952380952380952, 'number': 809}	{'precision': 0.12, 'recall': 0.05042016806722689, 'f1': 0.07100591715976332, 'number': 119}	{'precision': 0.6459369817578773, 'recall': 0.7314553990610329, 'f1': 0.6860413914575078, 'number': 1065}	0.6041	0.6448	0.6238	0.7497
0.7042	5.0	50	0.7333	{'precision': 0.628385698808234, 'recall': 0.7169344870210136, 'f1': 0.6697459584295612, 'number': 809}	{'precision': 0.29411764705882354, 'recall': 0.16806722689075632, 'f1': 0.21390374331550802, 'number': 119}	{'precision': 0.6616242038216561, 'recall': 0.780281690140845, 'f1': 0.7160706591986213, 'number': 1065}	0.6368	0.7180	0.6750	0.7748
0.6134	6.0	60	0.7075	{'precision': 0.6507276507276507, 'recall': 0.7737948084054388, 'f1': 0.7069452286843592, 'number': 809}	{'precision': 0.2987012987012987, 'recall': 0.19327731092436976, 'f1': 0.23469387755102045, 'number': 119}	{'precision': 0.7140366172624237, 'recall': 0.7690140845070422, 'f1': 0.7405063291139241, 'number': 1065}	0.6715	0.7366	0.7026	0.7789
0.5519	7.0	70	0.6817	{'precision': 0.6593521421107628, 'recall': 0.7799752781211372, 'f1': 0.7146092865232163, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.24369747899159663, 'f1': 0.2815533980582524, 'number': 119}	{'precision': 0.7023608768971332, 'recall': 0.7821596244131456, 'f1': 0.7401155042203464, 'number': 1065}	0.6695	0.7491	0.7071	0.7857
0.5105	8.0	80	0.6738	{'precision': 0.6628630705394191, 'recall': 0.7898640296662547, 'f1': 0.7208121827411168, 'number': 809}	{'precision': 0.2912621359223301, 'recall': 0.25210084033613445, 'f1': 0.2702702702702703, 'number': 119}	{'precision': 0.709106239460371, 'recall': 0.7896713615023474, 'f1': 0.7472234562416704, 'number': 1065}	0.6702	0.7577	0.7113	0.7899
0.4684	9.0	90	0.6721	{'precision': 0.6656217345872518, 'recall': 0.7873918417799752, 'f1': 0.7214043035107587, 'number': 809}	{'precision': 0.3090909090909091, 'recall': 0.2857142857142857, 'f1': 0.296943231441048, 'number': 119}	{'precision': 0.703150912106136, 'recall': 0.7962441314553991, 'f1': 0.7468075737560547, 'number': 1065}	0.6683	0.7622	0.7121	0.7906
0.4814	10.0	100	0.6712	{'precision': 0.6719409282700421, 'recall': 0.7873918417799752, 'f1': 0.7250996015936254, 'number': 809}	{'precision': 0.3153153153153153, 'recall': 0.29411764705882354, 'f1': 0.30434782608695654, 'number': 119}	{'precision': 0.7069109075770191, 'recall': 0.7971830985915493, 'f1': 0.7493380406001765, 'number': 1065}	0.6730	0.7632	0.7153	0.7909

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1