layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7094
• Answer: {'precision': 0.7131868131868132, 'recall': 0.8022249690976514, 'f1': 0.755090168702734, 'number': 809}
• Header: {'precision': 0.3445378151260504, 'recall': 0.3445378151260504, 'f1': 0.3445378151260504, 'number': 119}
• Question: {'precision': 0.7785651018600531, 'recall': 0.8253521126760563, 'f1': 0.8012762078395624, 'number': 1065}
• Overall Precision: 0.7271
• Overall Recall: 0.7873
• Overall F1: 0.7560
• Overall Accuracy: 0.8026

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7767	1.0	10	1.5683	{'precision': 0.021764032073310423, 'recall': 0.023485784919653894, 'f1': 0.022592152199762183, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.214987714987715, 'recall': 0.1643192488262911, 'f1': 0.18626929217668972, 'number': 1065}	0.1150	0.0973	0.1054	0.3768
1.4234	2.0	20	1.2196	{'precision': 0.1918194640338505, 'recall': 0.1681087762669963, 'f1': 0.17918313570487485, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4225037257824143, 'recall': 0.532394366197183, 'f1': 0.47112588284171164, 'number': 1065}	0.3409	0.3527	0.3467	0.5773
1.0839	3.0	30	0.9585	{'precision': 0.4686390532544379, 'recall': 0.4894932014833127, 'f1': 0.4788391777509069, 'number': 809}	{'precision': 0.13043478260869565, 'recall': 0.05042016806722689, 'f1': 0.07272727272727272, 'number': 119}	{'precision': 0.5354637568199533, 'recall': 0.6450704225352113, 'f1': 0.5851788756388415, 'number': 1065}	0.5009	0.5464	0.5227	0.7008
0.8429	4.0	40	0.8025	{'precision': 0.6150583244962884, 'recall': 0.7169344870210136, 'f1': 0.6621004566210046, 'number': 809}	{'precision': 0.32142857142857145, 'recall': 0.15126050420168066, 'f1': 0.20571428571428574, 'number': 119}	{'precision': 0.6584536958368734, 'recall': 0.7276995305164319, 'f1': 0.6913470115967887, 'number': 1065}	0.6310	0.6889	0.6587	0.7534
0.6591	5.0	50	0.7255	{'precision': 0.6464208242950108, 'recall': 0.7367119901112484, 'f1': 0.6886192952050838, 'number': 809}	{'precision': 0.25, 'recall': 0.19327731092436976, 'f1': 0.2180094786729858, 'number': 119}	{'precision': 0.6565891472868217, 'recall': 0.7953051643192488, 'f1': 0.7193205944798302, 'number': 1065}	0.6363	0.7356	0.6823	0.7769
0.5607	6.0	60	0.7110	{'precision': 0.6417759838546923, 'recall': 0.7861557478368356, 'f1': 0.7066666666666668, 'number': 809}	{'precision': 0.30337078651685395, 'recall': 0.226890756302521, 'f1': 0.2596153846153846, 'number': 119}	{'precision': 0.7202432667245873, 'recall': 0.7784037558685446, 'f1': 0.7481949458483754, 'number': 1065}	0.6688	0.7486	0.7064	0.7806
0.483	7.0	70	0.6787	{'precision': 0.6635120925341745, 'recall': 0.7799752781211372, 'f1': 0.7170454545454545, 'number': 809}	{'precision': 0.2777777777777778, 'recall': 0.25210084033613445, 'f1': 0.2643171806167401, 'number': 119}	{'precision': 0.7391688770999116, 'recall': 0.7849765258215963, 'f1': 0.761384335154827, 'number': 1065}	0.6836	0.7511	0.7158	0.7923
0.4275	8.0	80	0.6793	{'precision': 0.6615067079463365, 'recall': 0.792336217552534, 'f1': 0.7210348706411699, 'number': 809}	{'precision': 0.29906542056074764, 'recall': 0.2689075630252101, 'f1': 0.28318584070796454, 'number': 119}	{'precision': 0.7489139878366637, 'recall': 0.8093896713615023, 'f1': 0.7779783393501806, 'number': 1065}	0.6893	0.7702	0.7275	0.7970
0.3762	9.0	90	0.6784	{'precision': 0.6949516648764769, 'recall': 0.799752781211372, 'f1': 0.7436781609195402, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.3277310924369748, 'f1': 0.3305084745762712, 'number': 119}	{'precision': 0.7506493506493507, 'recall': 0.8140845070422535, 'f1': 0.781081081081081, 'number': 1065}	0.7049	0.7792	0.7402	0.8000
0.3634	10.0	100	0.6793	{'precision': 0.6964477933261571, 'recall': 0.799752781211372, 'f1': 0.7445339470655927, 'number': 809}	{'precision': 0.375, 'recall': 0.3277310924369748, 'f1': 0.3497757847533633, 'number': 119}	{'precision': 0.7650655021834061, 'recall': 0.8225352112676056, 'f1': 0.7927601809954752, 'number': 1065}	0.7172	0.7837	0.7490	0.8033
0.3104	11.0	110	0.6977	{'precision': 0.694327731092437, 'recall': 0.8170580964153276, 'f1': 0.750709823963657, 'number': 809}	{'precision': 0.33613445378151263, 'recall': 0.33613445378151263, 'f1': 0.33613445378151263, 'number': 119}	{'precision': 0.7766903914590747, 'recall': 0.819718309859155, 'f1': 0.7976244860666972, 'number': 1065}	0.7171	0.7898	0.7517	0.8032
0.2928	12.0	120	0.6987	{'precision': 0.6931330472103004, 'recall': 0.7985166872682324, 'f1': 0.7421022400919012, 'number': 809}	{'precision': 0.4, 'recall': 0.35294117647058826, 'f1': 0.37500000000000006, 'number': 119}	{'precision': 0.7809439002671416, 'recall': 0.8234741784037559, 'f1': 0.8016453382084096, 'number': 1065}	0.7245	0.7852	0.7537	0.8028
0.2766	13.0	130	0.7057	{'precision': 0.6996770721205597, 'recall': 0.8034610630407911, 'f1': 0.7479861910241656, 'number': 809}	{'precision': 0.3277310924369748, 'recall': 0.3277310924369748, 'f1': 0.3277310924369748, 'number': 119}	{'precision': 0.7749338040600177, 'recall': 0.8244131455399061, 'f1': 0.7989080982711555, 'number': 1065}	0.7185	0.7863	0.7508	0.8031
0.2627	14.0	140	0.7089	{'precision': 0.7063318777292577, 'recall': 0.799752781211372, 'f1': 0.750144927536232, 'number': 809}	{'precision': 0.3652173913043478, 'recall': 0.35294117647058826, 'f1': 0.35897435897435903, 'number': 119}	{'precision': 0.7798408488063661, 'recall': 0.828169014084507, 'f1': 0.8032786885245902, 'number': 1065}	0.7266	0.7883	0.7562	0.8012
0.2561	15.0	150	0.7094	{'precision': 0.7131868131868132, 'recall': 0.8022249690976514, 'f1': 0.755090168702734, 'number': 809}	{'precision': 0.3445378151260504, 'recall': 0.3445378151260504, 'f1': 0.3445378151260504, 'number': 119}	{'precision': 0.7785651018600531, 'recall': 0.8253521126760563, 'f1': 0.8012762078395624, 'number': 1065}	0.7271	0.7873	0.7560	0.8026

Framework versions

• Transformers 4.44.0
• Pytorch 2.2.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1