Hugging Face's logo

T-Brockhouse
/
lilt-en-funsd

Token Classification
Transformers
PyTorch
TensorBoard
Safetensors
lilt
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
T-Brockhouse commited on
Commit
6f541cb
·
1 Parent(s): 8c31d48

End of training

Browse files
Files changed (4) hide show
  1. README.md +25 -24
  2. logs/events.out.tfevents.1699273959.6e757f4abd68.3488.0 +2 -2
  3. logs/events.out.tfevents.1699275390.6e757f4abd68.3488.1 +3 -0
  4. model.safetensors +1 -1
README.md CHANGED
@@ -17,14 +17,14 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on the funsd-layoutlmv3 dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 1.7029
21
- - Answer: {'precision': 0.8619489559164734, 'recall': 0.9094247246022031, 'f1': 0.8850506253722453, 'number': 817}
22
- - Header: {'precision': 0.648936170212766, 'recall': 0.5126050420168067, 'f1': 0.5727699530516431, 'number': 119}
23
- - Question: {'precision': 0.8972046889089269, 'recall': 0.9238625812441968, 'f1': 0.9103385178408051, 'number': 1077}
24
- - Overall Precision: 0.8712
25
- - Overall Recall: 0.8937
26
- - Overall F1: 0.8823
27
- - Overall Accuracy: 0.8004
28
 
29
  ## Model description
30
 
@@ -50,28 +50,29 @@ The following hyperparameters were used during training:
50
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
  - lr_scheduler_type: linear
52
  - training_steps: 2500
 
53
 
54
  ### Training results
55
 
56
- | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
57
- |:-------------:|:------:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
58
- | 0.4178 | 10.53 | 200 | 1.1592 | {'precision': 0.8244444444444444, 'recall': 0.9082007343941249, 'f1': 0.8642981945253349, 'number': 817} | {'precision': 0.62, 'recall': 0.5210084033613446, 'f1': 0.5662100456621004, 'number': 119} | {'precision': 0.874561403508772, 'recall': 0.9257195914577531, 'f1': 0.8994136220117277, 'number': 1077} | 0.8416 | 0.8947 | 0.8673 | 0.7757 |
59
- | 0.0458 | 21.05 | 400 | 1.3058 | {'precision': 0.8473988439306358, 'recall': 0.8971848225214198, 'f1': 0.8715814506539833, 'number': 817} | {'precision': 0.5462184873949579, 'recall': 0.5462184873949579, 'f1': 0.5462184873949579, 'number': 119} | {'precision': 0.8808243727598566, 'recall': 0.9127205199628597, 'f1': 0.8964888280893752, 'number': 1077} | 0.8481 | 0.8847 | 0.8660 | 0.7946 |
60
- | 0.0135 | 31.58 | 600 | 1.6170 | {'precision': 0.855188141391106, 'recall': 0.9179926560587516, 'f1': 0.885478158205431, 'number': 817} | {'precision': 0.5737704918032787, 'recall': 0.5882352941176471, 'f1': 0.5809128630705394, 'number': 119} | {'precision': 0.8915223336371924, 'recall': 0.9080779944289693, 'f1': 0.8997240110395583, 'number': 1077} | 0.8578 | 0.8932 | 0.8752 | 0.7978 |
61
- | 0.0081 | 42.11 | 800 | 1.3449 | {'precision': 0.8650602409638555, 'recall': 0.8788249694002448, 'f1': 0.8718882817243473, 'number': 817} | {'precision': 0.6476190476190476, 'recall': 0.5714285714285714, 'f1': 0.6071428571428571, 'number': 119} | {'precision': 0.8675324675324675, 'recall': 0.9303621169916435, 'f1': 0.8978494623655914, 'number': 1077} | 0.8555 | 0.8882 | 0.8716 | 0.8073 |
62
- | 0.0048 | 52.63 | 1000 | 1.4197 | {'precision': 0.8709288299155609, 'recall': 0.8837209302325582, 'f1': 0.8772782503037667, 'number': 817} | {'precision': 0.5925925925925926, 'recall': 0.5378151260504201, 'f1': 0.5638766519823789, 'number': 119} | {'precision': 0.8701298701298701, 'recall': 0.9331476323119777, 'f1': 0.9005376344086022, 'number': 1077} | 0.8561 | 0.8897 | 0.8726 | 0.8155 |
63
- | 0.0021 | 63.16 | 1200 | 1.5708 | {'precision': 0.8366666666666667, 'recall': 0.9216646266829865, 'f1': 0.8771112405358183, 'number': 817} | {'precision': 0.6590909090909091, 'recall': 0.48739495798319327, 'f1': 0.5603864734299517, 'number': 119} | {'precision': 0.8841628959276018, 'recall': 0.9071494893221913, 'f1': 0.8955087076076994, 'number': 1077} | 0.8543 | 0.8882 | 0.8709 | 0.8014 |
64
- | 0.0019 | 73.68 | 1400 | 1.5995 | {'precision': 0.8174946004319654, 'recall': 0.9265605875152999, 'f1': 0.8686173264486519, 'number': 817} | {'precision': 0.5892857142857143, 'recall': 0.5546218487394958, 'f1': 0.5714285714285715, 'number': 119} | {'precision': 0.8880800727934486, 'recall': 0.9062209842154132, 'f1': 0.8970588235294117, 'number': 1077} | 0.8418 | 0.8937 | 0.8670 | 0.8108 |
65
- | 0.0015 | 84.21 | 1600 | 1.6443 | {'precision': 0.8770883054892601, 'recall': 0.8996328029375765, 'f1': 0.8882175226586103, 'number': 817} | {'precision': 0.5945945945945946, 'recall': 0.5546218487394958, 'f1': 0.5739130434782609, 'number': 119} | {'precision': 0.8947368421052632, 'recall': 0.9155060352831941, 'f1': 0.9050022946305646, 'number': 1077} | 0.8713 | 0.8877 | 0.8794 | 0.8002 |
66
- | 0.0011 | 94.74 | 1800 | 1.6845 | {'precision': 0.8622685185185185, 'recall': 0.9118727050183598, 'f1': 0.8863771564544913, 'number': 817} | {'precision': 0.6444444444444445, 'recall': 0.48739495798319327, 'f1': 0.5550239234449761, 'number': 119} | {'precision': 0.8973660308810173, 'recall': 0.9173630454967502, 'f1': 0.9072543617998163, 'number': 1077} | 0.8715 | 0.8897 | 0.8805 | 0.7996 |
67
- | 0.0003 | 105.26 | 2000 | 1.6754 | {'precision': 0.8582949308755761, 'recall': 0.9118727050183598, 'f1': 0.8842729970326408, 'number': 817} | {'precision': 0.5961538461538461, 'recall': 0.5210084033613446, 'f1': 0.5560538116591929, 'number': 119} | {'precision': 0.9006381039197813, 'recall': 0.9173630454967502, 'f1': 0.9089236430542779, 'number': 1077} | 0.8676 | 0.8917 | 0.8795 | 0.8034 |
68
- | 0.0003 | 115.79 | 2200 | 1.6803 | {'precision': 0.8733727810650888, 'recall': 0.9033047735618115, 'f1': 0.888086642599278, 'number': 817} | {'precision': 0.6116504854368932, 'recall': 0.5294117647058824, 'f1': 0.5675675675675675, 'number': 119} | {'precision': 0.8938294010889292, 'recall': 0.914577530176416, 'f1': 0.9040844424047729, 'number': 1077} | 0.8712 | 0.8872 | 0.8792 | 0.8014 |
69
- | 0.0004 | 126.32 | 2400 | 1.7029 | {'precision': 0.8619489559164734, 'recall': 0.9094247246022031, 'f1': 0.8850506253722453, 'number': 817} | {'precision': 0.648936170212766, 'recall': 0.5126050420168067, 'f1': 0.5727699530516431, 'number': 119} | {'precision': 0.8972046889089269, 'recall': 0.9238625812441968, 'f1': 0.9103385178408051, 'number': 1077} | 0.8712 | 0.8937 | 0.8823 | 0.8004 |
70
 
71
 
72
  ### Framework versions
73
 
74
- - Transformers 4.34.1
75
  - Pytorch 2.1.0+cu118
76
- - Datasets 2.14.5
77
  - Tokenizers 0.14.1
 
17
 
18
  This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on the funsd-layoutlmv3 dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.6853
21
+ - Answer: {'precision': 0.8719723183391004, 'recall': 0.9253365973072215, 'f1': 0.8978622327790974, 'number': 817}
22
+ - Header: {'precision': 0.6224489795918368, 'recall': 0.5126050420168067, 'f1': 0.5622119815668203, 'number': 119}
23
+ - Question: {'precision': 0.908411214953271, 'recall': 0.9025069637883009, 'f1': 0.9054494643688868, 'number': 1077}
24
+ - Overall Precision: 0.8791
25
+ - Overall Recall: 0.8887
26
+ - Overall F1: 0.8839
27
+ - Overall Accuracy: 0.8067
28
 
29
  ## Model description
30
 
 
50
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
  - lr_scheduler_type: linear
52
  - training_steps: 2500
53
+ - mixed_precision_training: Native AMP
54
 
55
  ### Training results
56
 
57
+ | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
58
+ |:-------------:|:------:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
59
+ | 0.4358 | 10.53 | 200 | 1.0235 | {'precision': 0.8292964244521338, 'recall': 0.8800489596083231, 'f1': 0.8539192399049881, 'number': 817} | {'precision': 0.4657534246575342, 'recall': 0.5714285714285714, 'f1': 0.5132075471698114, 'number': 119} | {'precision': 0.8694852941176471, 'recall': 0.8783658310120706, 'f1': 0.8739030023094689, 'number': 1077} | 0.8248 | 0.8609 | 0.8425 | 0.7868 |
60
+ | 0.0513 | 21.05 | 400 | 1.2438 | {'precision': 0.8439306358381503, 'recall': 0.8935128518971848, 'f1': 0.8680142687277052, 'number': 817} | {'precision': 0.7045454545454546, 'recall': 0.5210084033613446, 'f1': 0.5990338164251209, 'number': 119} | {'precision': 0.8937893789378938, 'recall': 0.9220055710306406, 'f1': 0.9076782449725778, 'number': 1077} | 0.8648 | 0.8867 | 0.8756 | 0.8066 |
61
+ | 0.0139 | 31.58 | 600 | 1.3473 | {'precision': 0.8443935926773455, 'recall': 0.9033047735618115, 'f1': 0.872856298048492, 'number': 817} | {'precision': 0.6111111111111112, 'recall': 0.5546218487394958, 'f1': 0.5814977973568282, 'number': 119} | {'precision': 0.8945945945945946, 'recall': 0.9220055710306406, 'f1': 0.9080932784636488, 'number': 1077} | 0.8590 | 0.8927 | 0.8755 | 0.8101 |
62
+ | 0.0087 | 42.11 | 800 | 1.3432 | {'precision': 0.8778718258766627, 'recall': 0.8886168910648715, 'f1': 0.8832116788321168, 'number': 817} | {'precision': 0.5813953488372093, 'recall': 0.6302521008403361, 'f1': 0.6048387096774193, 'number': 119} | {'precision': 0.9113573407202216, 'recall': 0.9164345403899722, 'f1': 0.9138888888888889, 'number': 1077} | 0.8769 | 0.8882 | 0.8825 | 0.8161 |
63
+ | 0.0039 | 52.63 | 1000 | 1.5068 | {'precision': 0.8678362573099415, 'recall': 0.9082007343941249, 'f1': 0.8875598086124402, 'number': 817} | {'precision': 0.5564516129032258, 'recall': 0.5798319327731093, 'f1': 0.5679012345679013, 'number': 119} | {'precision': 0.8998144712430427, 'recall': 0.9006499535747446, 'f1': 0.9002320185614848, 'number': 1077} | 0.8658 | 0.8847 | 0.8752 | 0.8028 |
64
+ | 0.0028 | 63.16 | 1200 | 1.5721 | {'precision': 0.8624277456647399, 'recall': 0.9130966952264382, 'f1': 0.8870392390011891, 'number': 817} | {'precision': 0.6288659793814433, 'recall': 0.5126050420168067, 'f1': 0.5648148148148148, 'number': 119} | {'precision': 0.9085714285714286, 'recall': 0.8857938718662952, 'f1': 0.8970380818053596, 'number': 1077} | 0.8752 | 0.8748 | 0.8750 | 0.8145 |
65
+ | 0.0027 | 73.68 | 1400 | 1.5657 | {'precision': 0.8695150115473441, 'recall': 0.9216646266829865, 'f1': 0.8948306595365418, 'number': 817} | {'precision': 0.6310679611650486, 'recall': 0.5462184873949579, 'f1': 0.5855855855855856, 'number': 119} | {'precision': 0.8944494995450409, 'recall': 0.9127205199628597, 'f1': 0.9034926470588234, 'number': 1077} | 0.8709 | 0.8947 | 0.8826 | 0.8130 |
66
+ | 0.0012 | 84.21 | 1600 | 1.6853 | {'precision': 0.8719723183391004, 'recall': 0.9253365973072215, 'f1': 0.8978622327790974, 'number': 817} | {'precision': 0.6224489795918368, 'recall': 0.5126050420168067, 'f1': 0.5622119815668203, 'number': 119} | {'precision': 0.908411214953271, 'recall': 0.9025069637883009, 'f1': 0.9054494643688868, 'number': 1077} | 0.8791 | 0.8887 | 0.8839 | 0.8067 |
67
+ | 0.0007 | 94.74 | 1800 | 1.6321 | {'precision': 0.8642117376294591, 'recall': 0.9192166462668299, 'f1': 0.8908659549228943, 'number': 817} | {'precision': 0.5964912280701754, 'recall': 0.5714285714285714, 'f1': 0.5836909871244635, 'number': 119} | {'precision': 0.9101964452759589, 'recall': 0.903435468895079, 'f1': 0.9068033550792172, 'number': 1077} | 0.8733 | 0.8902 | 0.8817 | 0.8045 |
68
+ | 0.0004 | 105.26 | 2000 | 1.7732 | {'precision': 0.8535469107551488, 'recall': 0.9130966952264382, 'f1': 0.8823181549379067, 'number': 817} | {'precision': 0.5752212389380531, 'recall': 0.5462184873949579, 'f1': 0.5603448275862069, 'number': 119} | {'precision': 0.8991825613079019, 'recall': 0.9192200557103064, 'f1': 0.909090909090909, 'number': 1077} | 0.8625 | 0.8947 | 0.8783 | 0.7991 |
69
+ | 0.0003 | 115.79 | 2200 | 1.7988 | {'precision': 0.8785714285714286, 'recall': 0.9033047735618115, 'f1': 0.8907664453832227, 'number': 817} | {'precision': 0.6190476190476191, 'recall': 0.5462184873949579, 'f1': 0.5803571428571429, 'number': 119} | {'precision': 0.8940639269406393, 'recall': 0.9090064995357474, 'f1': 0.9014732965009209, 'number': 1077} | 0.8735 | 0.8852 | 0.8793 | 0.7950 |
70
+ | 0.0003 | 126.32 | 2400 | 1.8038 | {'precision': 0.8584686774941995, 'recall': 0.9057527539779682, 'f1': 0.8814770696843359, 'number': 817} | {'precision': 0.63, 'recall': 0.5294117647058824, 'f1': 0.5753424657534247, 'number': 119} | {'precision': 0.8943533697632058, 'recall': 0.9117920148560817, 'f1': 0.9029885057471265, 'number': 1077} | 0.8665 | 0.8867 | 0.8765 | 0.7953 |
71
 
72
 
73
  ### Framework versions
74
 
75
+ - Transformers 4.35.0
76
  - Pytorch 2.1.0+cu118
77
+ - Datasets 2.14.6
78
  - Tokenizers 0.14.1
logs/events.out.tfevents.1699273959.6e757f4abd68.3488.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5778faf9d0e66d070e035026c9510658aac768a15730e0f12fedb2813e408bcd
3
- size 12538
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17f888c6877d0d73037f2da5eec3cec6d89c39e5775a3c1bd2e4a44108a12013
3
+ size 12892
logs/events.out.tfevents.1699275390.6e757f4abd68.3488.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1cef6706ae300346b636f14cc2c6b5fbcb89d3c886a23a6ae037c7269b943e00
3
+ size 592
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:93ad4edf0bbe90090d4df9772c5a432e5aa7cbddafc0fa4873e4678442765adf
3
  size 520727564
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b12f981b59e02d39b06b15e53cad822cee93b5a43b56559e33934a99217ff2f
3
  size 520727564