layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6573
• Answer: {'precision': 0.7060773480662983, 'recall': 0.7898640296662547, 'f1': 0.7456242707117853, 'number': 809}
• Header: {'precision': 0.3333333333333333, 'recall': 0.3697478991596639, 'f1': 0.350597609561753, 'number': 119}
• Question: {'precision': 0.7687661777394306, 'recall': 0.8366197183098592, 'f1': 0.8012589928057554, 'number': 1065}
• Overall Precision: 0.7168
• Overall Recall: 0.7898
• Overall F1: 0.7515
• Overall Accuracy: 0.8172

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7999	1.0	10	1.5802	{'precision': 0.008905852417302799, 'recall': 0.00865265760197775, 'f1': 0.00877742946708464, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.1717325227963526, 'recall': 0.10610328638497653, 'f1': 0.13116656993615786, 'number': 1065}	0.0831	0.0602	0.0698	0.3604
1.4567	2.0	20	1.2493	{'precision': 0.18839103869653767, 'recall': 0.22867737948084055, 'f1': 0.20658849804578447, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.45693950177935944, 'recall': 0.6028169014084507, 'f1': 0.5198380566801619, 'number': 1065}	0.3465	0.4150	0.3776	0.5986
1.114	3.0	30	0.9406	{'precision': 0.43853820598006643, 'recall': 0.4894932014833127, 'f1': 0.46261682242990654, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5861538461538461, 'recall': 0.7154929577464789, 'f1': 0.6443974630021141, 'number': 1065}	0.5237	0.5810	0.5509	0.7001
0.8434	4.0	40	0.7906	{'precision': 0.5922836287799792, 'recall': 0.7021013597033374, 'f1': 0.6425339366515838, 'number': 809}	{'precision': 0.1111111111111111, 'recall': 0.04201680672268908, 'f1': 0.06097560975609755, 'number': 119}	{'precision': 0.6526994359387591, 'recall': 0.7605633802816901, 'f1': 0.7025151777970512, 'number': 1065}	0.6160	0.6939	0.6527	0.7541
0.6817	5.0	50	0.7106	{'precision': 0.6502192982456141, 'recall': 0.7330037082818294, 'f1': 0.6891342242882045, 'number': 809}	{'precision': 0.25301204819277107, 'recall': 0.17647058823529413, 'f1': 0.20792079207920794, 'number': 119}	{'precision': 0.683921568627451, 'recall': 0.8187793427230047, 'f1': 0.7452991452991454, 'number': 1065}	0.6546	0.7456	0.6972	0.7854
0.5737	6.0	60	0.6807	{'precision': 0.6482617586912065, 'recall': 0.7836835599505563, 'f1': 0.7095691102406267, 'number': 809}	{'precision': 0.273972602739726, 'recall': 0.16806722689075632, 'f1': 0.20833333333333331, 'number': 119}	{'precision': 0.717206132879046, 'recall': 0.7906103286384977, 'f1': 0.7521214828048235, 'number': 1065}	0.6724	0.7506	0.7093	0.7898
0.5058	7.0	70	0.6538	{'precision': 0.6564102564102564, 'recall': 0.7911001236093943, 'f1': 0.7174887892376681, 'number': 809}	{'precision': 0.3048780487804878, 'recall': 0.21008403361344538, 'f1': 0.24875621890547264, 'number': 119}	{'precision': 0.7324894514767932, 'recall': 0.8150234741784037, 'f1': 0.7715555555555556, 'number': 1065}	0.6838	0.7692	0.7240	0.7996
0.4425	8.0	80	0.6574	{'precision': 0.6625766871165644, 'recall': 0.8009888751545118, 'f1': 0.7252378287632905, 'number': 809}	{'precision': 0.3055555555555556, 'recall': 0.2773109243697479, 'f1': 0.2907488986784141, 'number': 119}	{'precision': 0.7365771812080537, 'recall': 0.8244131455399061, 'f1': 0.7780239255649092, 'number': 1065}	0.6844	0.7822	0.7300	0.7999
0.3932	9.0	90	0.6375	{'precision': 0.6876971608832808, 'recall': 0.8084054388133498, 'f1': 0.7431818181818182, 'number': 809}	{'precision': 0.3645833333333333, 'recall': 0.29411764705882354, 'f1': 0.3255813953488372, 'number': 119}	{'precision': 0.752129471890971, 'recall': 0.8291079812206573, 'f1': 0.7887449754354622, 'number': 1065}	0.7078	0.7888	0.7461	0.8087
0.3798	10.0	100	0.6437	{'precision': 0.6981541802388708, 'recall': 0.7948084054388134, 'f1': 0.7433526011560695, 'number': 809}	{'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119}	{'precision': 0.7665505226480837, 'recall': 0.8262910798122066, 'f1': 0.7953004970628107, 'number': 1065}	0.7136	0.7837	0.7470	0.8098
0.3225	11.0	110	0.6566	{'precision': 0.6817226890756303, 'recall': 0.8022249690976514, 'f1': 0.7370812038614423, 'number': 809}	{'precision': 0.336, 'recall': 0.35294117647058826, 'f1': 0.3442622950819672, 'number': 119}	{'precision': 0.7593856655290102, 'recall': 0.8356807511737089, 'f1': 0.7957085382208315, 'number': 1065}	0.7030	0.7933	0.7454	0.8038
0.3097	12.0	120	0.6421	{'precision': 0.6957928802588996, 'recall': 0.7972805933250927, 'f1': 0.7430875576036866, 'number': 809}	{'precision': 0.35, 'recall': 0.35294117647058826, 'f1': 0.35146443514644354, 'number': 119}	{'precision': 0.7692307692307693, 'recall': 0.8356807511737089, 'f1': 0.8010801080108011, 'number': 1065}	0.7155	0.7913	0.7515	0.8177
0.2916	13.0	130	0.6515	{'precision': 0.7035010940919038, 'recall': 0.7948084054388134, 'f1': 0.7463726059199072, 'number': 809}	{'precision': 0.33076923076923076, 'recall': 0.36134453781512604, 'f1': 0.34538152610441764, 'number': 119}	{'precision': 0.7649092480553155, 'recall': 0.8309859154929577, 'f1': 0.7965796579657966, 'number': 1065}	0.7138	0.7883	0.7492	0.8154
0.2707	14.0	140	0.6557	{'precision': 0.7016393442622951, 'recall': 0.7935723114956736, 'f1': 0.7447795823665894, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.36134453781512604, 'f1': 0.34677419354838707, 'number': 119}	{'precision': 0.7688966116420504, 'recall': 0.8309859154929577, 'f1': 0.7987364620938627, 'number': 1065}	0.7153	0.7878	0.7498	0.8146
0.2729	15.0	150	0.6573	{'precision': 0.7060773480662983, 'recall': 0.7898640296662547, 'f1': 0.7456242707117853, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.3697478991596639, 'f1': 0.350597609561753, 'number': 119}	{'precision': 0.7687661777394306, 'recall': 0.8366197183098592, 'f1': 0.8012589928057554, 'number': 1065}	0.7168	0.7898	0.7515	0.8172

Framework versions

• Transformers 4.40.1
• Pytorch 2.3.0+cpu
• Datasets 2.19.0
• Tokenizers 0.19.1