lmv2-g-rai-auth-02-14

This model is a fine-tuned version of microsoft/layoutlmv2-base-uncased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0368
  • Dob Key Precision: 0.5057
  • Dob Key Recall: 0.5205
  • Dob Key F1: 0.5130
  • Dob Key Number: 171
  • Dob Value Precision: 0.8071
  • Dob Value Recall: 0.9191
  • Dob Value F1: 0.8595
  • Dob Value Number: 173
  • Patient Name Key Precision: 0.6923
  • Patient Name Key Recall: 0.7219
  • Patient Name Key F1: 0.7068
  • Patient Name Key Number: 187
  • Patient Name Value Precision: 0.9235
  • Patient Name Value Recall: 0.9628
  • Patient Name Value F1: 0.9427
  • Patient Name Value Number: 188
  • Provider Name Key Precision: 0.6930
  • Provider Name Key Recall: 0.7065
  • Provider Name Key F1: 0.6997
  • Provider Name Key Number: 460
  • Provider Name Value Precision: 0.9353
  • Provider Name Value Recall: 0.9476
  • Provider Name Value F1: 0.9414
  • Provider Name Value Number: 458
  • Overall Precision: 0.7796
  • Overall Recall: 0.8082
  • Overall F1: 0.7936
  • Overall Accuracy: 0.9944

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Dob Key Precision Dob Key Recall Dob Key F1 Dob Key Number Dob Value Precision Dob Value Recall Dob Value F1 Dob Value Number Patient Name Key Precision Patient Name Key Recall Patient Name Key F1 Patient Name Key Number Patient Name Value Precision Patient Name Value Recall Patient Name Value F1 Patient Name Value Number Provider Name Key Precision Provider Name Key Recall Provider Name Key F1 Provider Name Key Number Provider Name Value Precision Provider Name Value Recall Provider Name Value F1 Provider Name Value Number Overall Precision Overall Recall Overall F1 Overall Accuracy
1.1221 1.0 241 0.4373 0.0 0.0 0.0 171 0.0 0.0 0.0 173 0.0 0.0 0.0 187 0.0 0.0 0.0 188 0.0 0.0 0.0 460 0.0 0.0 0.0 458 0.0 0.0 0.0 0.9696
0.258 2.0 482 0.1408 0.0385 0.0351 0.0367 171 0.9778 0.2543 0.4037 173 0.0385 0.0053 0.0094 187 0.1739 0.0426 0.0684 188 0.0286 0.0043 0.0075 460 0.6628 0.7424 0.7003 458 0.4685 0.2450 0.3217 0.9782
0.1066 3.0 723 0.0774 0.4011 0.4386 0.4190 171 0.8404 0.9133 0.8753 173 0.5097 0.5615 0.5344 187 0.4804 0.7181 0.5757 188 0.5108 0.5674 0.5376 460 0.8841 0.9323 0.9075 458 0.6255 0.7092 0.6648 0.9920
0.0685 4.0 964 0.0585 0.4229 0.4327 0.4277 171 0.8495 0.9133 0.8802 173 0.5479 0.5508 0.5493 187 0.9005 0.9628 0.9306 188 0.6362 0.6957 0.6646 460 0.9315 0.9498 0.9405 458 0.7390 0.7764 0.7572 0.9938
0.0532 5.0 1205 0.0486 0.4432 0.4561 0.4496 171 0.8634 0.9133 0.8876 173 0.6862 0.6898 0.688 187 0.905 0.9628 0.9330 188 0.7106 0.7152 0.7129 460 0.9375 0.9498 0.9436 458 0.7826 0.8002 0.7913 0.9943
0.0453 6.0 1446 0.0429 0.4277 0.4327 0.4302 171 0.8971 0.9075 0.9023 173 0.6806 0.6952 0.6878 187 0.8835 0.9681 0.9239 188 0.7181 0.7087 0.7133 460 0.9332 0.9454 0.9393 458 0.7829 0.7954 0.7891 0.9943
0.0392 7.0 1687 0.0392 0.4432 0.4561 0.4496 171 0.8177 0.9075 0.8603 173 0.6875 0.7059 0.6966 187 0.9333 0.9681 0.9504 188 0.7045 0.7152 0.7098 460 0.9353 0.9476 0.9414 458 0.7782 0.8015 0.7896 0.9944
0.0351 8.0 1928 0.0368 0.5057 0.5205 0.5130 171 0.8071 0.9191 0.8595 173 0.6923 0.7219 0.7068 187 0.9235 0.9628 0.9427 188 0.6930 0.7065 0.6997 460 0.9353 0.9476 0.9414 458 0.7796 0.8082 0.7936 0.9944
0.0326 9.0 2169 0.0354 0.4375 0.4503 0.4438 171 0.8438 0.9364 0.8877 173 0.6943 0.7166 0.7053 187 0.9235 0.9628 0.9427 188 0.7063 0.7109 0.7086 460 0.9353 0.9476 0.9414 458 0.7809 0.8033 0.7919 0.9944
0.0313 10.0 2410 0.0350 0.4886 0.5029 0.4957 171 0.8777 0.9538 0.9141 173 0.6959 0.7219 0.7087 187 0.9188 0.9628 0.9403 188 0.6674 0.7022 0.6843 460 0.9333 0.9476 0.9404 458 0.7770 0.8088 0.7926 0.9944

Framework versions

  • Transformers 4.27.0.dev0
  • Pytorch 1.13.1+cu116
  • Datasets 2.2.2
  • Tokenizers 0.13.2
Downloads last month
24
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.