--- library_name: transformers license: mit base_model: SCUT-DLVCLab/lilt-roberta-en-base tags: - generated_from_trainer model-index: - name: lilt-en-funsd results: [] --- # lilt-en-funsd This model is a fine-tuned version of [SCUT-DLVCLab/lilt-roberta-en-base](https://huggingface.co/SCUT-DLVCLab/lilt-roberta-en-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.7187 - Answer: {'precision': 0.8569767441860465, 'recall': 0.9020807833537332, 'f1': 0.8789505068574837, 'number': 817} - Header: {'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119} - Question: {'precision': 0.8962693357597816, 'recall': 0.914577530176416, 'f1': 0.9053308823529412, 'number': 1077} - Overall Precision: 0.8671 - Overall Recall: 0.8882 - Overall F1: 0.8775 - Overall Accuracy: 0.7998 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: linear - training_steps: 2500 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |:-------------:|:--------:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| | 0.4061 | 10.5263 | 200 | 1.0439 | {'precision': 0.8062770562770563, 'recall': 0.9118727050183598, 'f1': 0.855829982768524, 'number': 817} | {'precision': 0.6043956043956044, 'recall': 0.46218487394957986, 'f1': 0.5238095238095237, 'number': 119} | {'precision': 0.8777173913043478, 'recall': 0.8997214484679665, 'f1': 0.8885832187070151, 'number': 1077} | 0.8348 | 0.8788 | 0.8562 | 0.7914 | | 0.0454 | 21.0526 | 400 | 1.4836 | {'precision': 0.8243688254665203, 'recall': 0.9192166462668299, 'f1': 0.869212962962963, 'number': 817} | {'precision': 0.48872180451127817, 'recall': 0.5462184873949579, 'f1': 0.5158730158730158, 'number': 119} | {'precision': 0.9097963142580019, 'recall': 0.8709377901578459, 'f1': 0.889943074003795, 'number': 1077} | 0.8453 | 0.8713 | 0.8581 | 0.7934 | | 0.0145 | 31.5789 | 600 | 1.3593 | {'precision': 0.8672248803827751, 'recall': 0.8873929008567931, 'f1': 0.8771929824561404, 'number': 817} | {'precision': 0.6018518518518519, 'recall': 0.5462184873949579, 'f1': 0.5726872246696034, 'number': 119} | {'precision': 0.8753339269813001, 'recall': 0.9127205199628597, 'f1': 0.8936363636363636, 'number': 1077} | 0.8578 | 0.8808 | 0.8691 | 0.8041 | | 0.0074 | 42.1053 | 800 | 1.5465 | {'precision': 0.8311111111111111, 'recall': 0.9155446756425949, 'f1': 0.8712871287128713, 'number': 817} | {'precision': 0.6190476190476191, 'recall': 0.5462184873949579, 'f1': 0.5803571428571429, 'number': 119} | {'precision': 0.9031657355679702, 'recall': 0.9006499535747446, 'f1': 0.901906090190609, 'number': 1077} | 0.8576 | 0.8857 | 0.8715 | 0.7981 | | 0.0045 | 52.6316 | 1000 | 1.4018 | {'precision': 0.8342922899884925, 'recall': 0.8873929008567931, 'f1': 0.860023724792408, 'number': 817} | {'precision': 0.5803571428571429, 'recall': 0.5462184873949579, 'f1': 0.5627705627705628, 'number': 119} | {'precision': 0.8855855855855855, 'recall': 0.9127205199628597, 'f1': 0.8989483310470964, 'number': 1077} | 0.8479 | 0.8808 | 0.8640 | 0.8064 | | 0.0031 | 63.1579 | 1200 | 1.6815 | {'precision': 0.8714810281517748, 'recall': 0.8714810281517748, 'f1': 0.8714810281517748, 'number': 817} | {'precision': 0.6039603960396039, 'recall': 0.5126050420168067, 'f1': 0.5545454545454545, 'number': 119} | {'precision': 0.8730569948186528, 'recall': 0.9387186629526463, 'f1': 0.9046979865771813, 'number': 1077} | 0.8593 | 0.8862 | 0.8726 | 0.7952 | | 0.0018 | 73.6842 | 1400 | 1.5823 | {'precision': 0.8553530751708428, 'recall': 0.9192166462668299, 'f1': 0.8861356932153391, 'number': 817} | {'precision': 0.5865384615384616, 'recall': 0.5126050420168067, 'f1': 0.5470852017937219, 'number': 119} | {'precision': 0.8930530164533821, 'recall': 0.9071494893221913, 'f1': 0.9000460617227084, 'number': 1077} | 0.8618 | 0.8887 | 0.8750 | 0.8061 | | 0.0015 | 84.2105 | 1600 | 1.6540 | {'precision': 0.8509895227008148, 'recall': 0.8947368421052632, 'f1': 0.8723150357995225, 'number': 817} | {'precision': 0.6477272727272727, 'recall': 0.4789915966386555, 'f1': 0.5507246376811594, 'number': 119} | {'precision': 0.8776408450704225, 'recall': 0.9257195914577531, 'f1': 0.9010393131495708, 'number': 1077} | 0.8569 | 0.8867 | 0.8716 | 0.8039 | | 0.0005 | 94.7368 | 1800 | 1.7397 | {'precision': 0.8578199052132701, 'recall': 0.8861689106487148, 'f1': 0.8717639975918122, 'number': 817} | {'precision': 0.5740740740740741, 'recall': 0.5210084033613446, 'f1': 0.5462555066079295, 'number': 119} | {'precision': 0.8785971223021583, 'recall': 0.9071494893221913, 'f1': 0.8926450433988122, 'number': 1077} | 0.8542 | 0.8758 | 0.8649 | 0.7925 | | 0.0003 | 105.2632 | 2000 | 1.6680 | {'precision': 0.8688915375446961, 'recall': 0.8922888616891065, 'f1': 0.8804347826086957, 'number': 817} | {'precision': 0.6122448979591837, 'recall': 0.5042016806722689, 'f1': 0.5529953917050692, 'number': 119} | {'precision': 0.8774250440917107, 'recall': 0.9238625812441968, 'f1': 0.9000452284034374, 'number': 1077} | 0.8614 | 0.8862 | 0.8737 | 0.8011 | | 0.0002 | 115.7895 | 2200 | 1.6812 | {'precision': 0.8494252873563218, 'recall': 0.9045287637698899, 'f1': 0.8761114404267932, 'number': 817} | {'precision': 0.6704545454545454, 'recall': 0.4957983193277311, 'f1': 0.5700483091787439, 'number': 119} | {'precision': 0.8914798206278027, 'recall': 0.9229340761374187, 'f1': 0.906934306569343, 'number': 1077} | 0.8644 | 0.8902 | 0.8771 | 0.8051 | | 0.0004 | 126.3158 | 2400 | 1.7187 | {'precision': 0.8569767441860465, 'recall': 0.9020807833537332, 'f1': 0.8789505068574837, 'number': 817} | {'precision': 0.6407766990291263, 'recall': 0.5546218487394958, 'f1': 0.5945945945945947, 'number': 119} | {'precision': 0.8962693357597816, 'recall': 0.914577530176416, 'f1': 0.9053308823529412, 'number': 1077} | 0.8671 | 0.8882 | 0.8775 | 0.7998 | ### Framework versions - Transformers 4.46.2 - Pytorch 2.5.0+cu121 - Datasets 3.1.0 - Tokenizers 0.20.3