MeMo-BERT-WSD / README.md
yemen2016's picture
Update README.md
f466dd3 verified
metadata
base_model: MiMe-MeMo/MeMo-BERT-03
tags:
  - generated_from_trainer
model-index:
  - name: MeMo-BERT-WSD
    results: []
    language: da
widget:
  - text: >-
      Men havde Gud vendt sig fra ham , saa kunde han ogsaa vende sig fra Gud .
      Havde Gud ingen Øren , saa havde han heller ingen Læber , havde Gud ingen
      Naade , saa havde han heller ingen Tilbedelse , og han trodsede og viste
      Gud ud af sit Hjærte .

MeMo-BERT-WSD

This model is a fine-tuned version of MiMe-MeMo/MeMo-BERT-03 on https://huggingface.co/MiMe-MeMo/MeMo-Dataset-WSD dataset. It achieves the following results on the evaluation set:

  • Loss: 3.1503
  • F1-score: 0.5541

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss F1-score
No log 1.0 61 1.3445 0.2569
No log 2.0 122 1.0424 0.5124
No log 3.0 183 1.1609 0.5304
No log 4.0 244 1.3851 0.5389
No log 5.0 305 1.9822 0.4456
No log 6.0 366 2.0347 0.4914
No log 7.0 427 2.9891 0.4419
No log 8.0 488 2.5316 0.5183
0.4858 9.0 549 2.5900 0.5419
0.4858 10.0 610 2.9300 0.5051
0.4858 11.0 671 3.0018 0.5211
0.4858 12.0 732 3.0486 0.5109
0.4858 13.0 793 3.0887 0.5337
0.4858 14.0 854 3.1180 0.5441
0.4858 15.0 915 3.1503 0.5541
0.4858 16.0 976 3.1649 0.5436
0.0041 17.0 1037 3.1925 0.5436
0.0041 18.0 1098 3.2019 0.5436
0.0041 19.0 1159 3.2089 0.5436
0.0041 20.0 1220 3.2116 0.5436

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2