layoutlm-funsd-c

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7152
• Answer: {'precision': 0.7134894091415831, 'recall': 0.7911001236093943, 'f1': 0.7502930832356389, 'number': 809}
• Header: {'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119}
• Question: {'precision': 0.7805309734513274, 'recall': 0.828169014084507, 'f1': 0.8036446469248291, 'number': 1065}
• Overall Precision: 0.7245
• Overall Recall: 0.7837
• Overall F1: 0.7530
• Overall Accuracy: 0.8069

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7835	1.0	10	1.5696	{'precision': 0.02753303964757709, 'recall': 0.030902348578491966, 'f1': 0.029120559114735003, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.23644444444444446, 'recall': 0.24976525821596243, 'f1': 0.24292237442922376, 'number': 1065}	0.1431	0.1460	0.1446	0.4162
1.4134	2.0	20	1.2167	{'precision': 0.15942028985507245, 'recall': 0.13597033374536466, 'f1': 0.1467645096731154, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.42325227963525835, 'recall': 0.5230046948356808, 'f1': 0.4678706425871483, 'number': 1065}	0.3322	0.3347	0.3334	0.5768
1.0829	3.0	30	0.9351	{'precision': 0.4783599088838269, 'recall': 0.519159456118665, 'f1': 0.4979253112033195, 'number': 809}	{'precision': 0.034482758620689655, 'recall': 0.008403361344537815, 'f1': 0.013513513513513513, 'number': 119}	{'precision': 0.6103896103896104, 'recall': 0.6619718309859155, 'f1': 0.6351351351351351, 'number': 1065}	0.5461	0.5650	0.5554	0.7105
0.8077	4.0	40	0.7702	{'precision': 0.6122233930453108, 'recall': 0.7181705809641533, 'f1': 0.6609783845278726, 'number': 809}	{'precision': 0.2033898305084746, 'recall': 0.10084033613445378, 'f1': 0.13483146067415733, 'number': 119}	{'precision': 0.6381631037212985, 'recall': 0.7568075117370892, 'f1': 0.6924398625429553, 'number': 1065}	0.6160	0.7020	0.6562	0.7659
0.6407	5.0	50	0.7146	{'precision': 0.6491978609625668, 'recall': 0.7503090234857849, 'f1': 0.6961009174311926, 'number': 809}	{'precision': 0.2948717948717949, 'recall': 0.19327731092436976, 'f1': 0.233502538071066, 'number': 119}	{'precision': 0.6921221864951769, 'recall': 0.8084507042253521, 'f1': 0.7457773928107406, 'number': 1065}	0.6606	0.7481	0.7016	0.7869
0.5585	6.0	60	0.6995	{'precision': 0.673866090712743, 'recall': 0.7713226205191595, 'f1': 0.7193083573487031, 'number': 809}	{'precision': 0.3372093023255814, 'recall': 0.24369747899159663, 'f1': 0.2829268292682927, 'number': 119}	{'precision': 0.7374784110535406, 'recall': 0.8018779342723005, 'f1': 0.768331084120558, 'number': 1065}	0.6945	0.7561	0.7240	0.7948
0.4934	7.0	70	0.6852	{'precision': 0.6681222707423581, 'recall': 0.7564894932014833, 'f1': 0.7095652173913044, 'number': 809}	{'precision': 0.37777777777777777, 'recall': 0.2857142857142857, 'f1': 0.3253588516746411, 'number': 119}	{'precision': 0.7634408602150538, 'recall': 0.8, 'f1': 0.7812929848693261, 'number': 1065}	0.7059	0.7516	0.7281	0.7979
0.4384	8.0	80	0.6731	{'precision': 0.6920492721164614, 'recall': 0.7639060568603214, 'f1': 0.7262044653349001, 'number': 809}	{'precision': 0.3008130081300813, 'recall': 0.31092436974789917, 'f1': 0.3057851239669422, 'number': 119}	{'precision': 0.7508503401360545, 'recall': 0.8291079812206573, 'f1': 0.788041053101294, 'number': 1065}	0.7016	0.7717	0.7350	0.8021
0.3737	9.0	90	0.6766	{'precision': 0.6993392070484582, 'recall': 0.7849196538936959, 'f1': 0.7396622015142692, 'number': 809}	{'precision': 0.2992125984251969, 'recall': 0.31932773109243695, 'f1': 0.30894308943089427, 'number': 119}	{'precision': 0.7890974084003575, 'recall': 0.8291079812206573, 'f1': 0.8086080586080587, 'number': 1065}	0.7224	0.7807	0.7504	0.8046
0.341	10.0	100	0.6950	{'precision': 0.6888888888888889, 'recall': 0.7663782447466008, 'f1': 0.7255705090696314, 'number': 809}	{'precision': 0.3619047619047619, 'recall': 0.31932773109243695, 'f1': 0.33928571428571425, 'number': 119}	{'precision': 0.7859030837004405, 'recall': 0.8375586854460094, 'f1': 0.8109090909090909, 'number': 1065}	0.7243	0.7777	0.7501	0.8088
0.3178	11.0	110	0.6979	{'precision': 0.7157534246575342, 'recall': 0.7750309023485785, 'f1': 0.7442136498516321, 'number': 809}	{'precision': 0.375, 'recall': 0.35294117647058826, 'f1': 0.3636363636363636, 'number': 119}	{'precision': 0.7805092186128183, 'recall': 0.8347417840375587, 'f1': 0.8067150635208712, 'number': 1065}	0.7325	0.7817	0.7563	0.8059
0.2998	12.0	120	0.7019	{'precision': 0.7027624309392265, 'recall': 0.7861557478368356, 'f1': 0.7421236872812136, 'number': 809}	{'precision': 0.32061068702290074, 'recall': 0.35294117647058826, 'f1': 0.336, 'number': 119}	{'precision': 0.7885816235504014, 'recall': 0.8300469483568075, 'f1': 0.808783165599268, 'number': 1065}	0.7242	0.7837	0.7528	0.8069
0.2809	13.0	130	0.7056	{'precision': 0.7177777777777777, 'recall': 0.7985166872682324, 'f1': 0.7559976594499708, 'number': 809}	{'precision': 0.3565217391304348, 'recall': 0.3445378151260504, 'f1': 0.3504273504273504, 'number': 119}	{'precision': 0.7911504424778761, 'recall': 0.8394366197183099, 'f1': 0.8145785876993167, 'number': 1065}	0.7371	0.7933	0.7641	0.8097
0.2656	14.0	140	0.7117	{'precision': 0.718609865470852, 'recall': 0.792336217552534, 'f1': 0.7536743092298648, 'number': 809}	{'precision': 0.33884297520661155, 'recall': 0.3445378151260504, 'f1': 0.3416666666666667, 'number': 119}	{'precision': 0.7888198757763976, 'recall': 0.8347417840375587, 'f1': 0.8111313868613138, 'number': 1065}	0.7341	0.7883	0.7602	0.8098
0.2669	15.0	150	0.7152	{'precision': 0.7134894091415831, 'recall': 0.7911001236093943, 'f1': 0.7502930832356389, 'number': 809}	{'precision': 0.31007751937984496, 'recall': 0.33613445378151263, 'f1': 0.3225806451612903, 'number': 119}	{'precision': 0.7805309734513274, 'recall': 0.828169014084507, 'f1': 0.8036446469248291, 'number': 1065}	0.7245	0.7837	0.7530	0.8069

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0