Hugging Face's logo

DGurgurov
/
maltese-cn-lang-adapter

TensorBoard
Generated from Trainer
Model card Files Metrics Community
maltese-cn-lang-adapter / README.md
DGurgurov's picture
DGurgurov
Update README.md
731b6cb verified
preview code
|
raw
history blame
7.76 kB
metadata
license: apache-2.0
base_model: bert-base-multilingual-cased
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: mt
    results: []

mt

This model is an adapter fine-tuned on top of of bert-base-multilingual-cased on the Maltese ConceptNet dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8117
  • Accuracy: 0.8590

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • distributed_type: multi-GPU
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • training_steps: 50000

Training results

Training Loss Epoch Step Validation Loss Accuracy
2.6689 1.04 500 2.1345 0.6677
2.1415 2.07 1000 1.8436 0.6926
1.9421 3.11 1500 1.7874 0.6907
1.7588 4.14 2000 1.7605 0.7013
1.6729 5.18 2500 1.7568 0.6957
1.596 6.21 3000 1.5006 0.7273
1.5778 7.25 3500 1.3924 0.7451
1.4821 8.28 4000 1.6097 0.7099
1.4183 9.32 4500 1.3552 0.7491
1.4197 10.35 5000 1.2847 0.7513
1.3156 11.39 5500 1.3173 0.7496
1.2882 12.42 6000 1.2817 0.7738
1.2692 13.46 6500 1.1892 0.7751
1.2368 14.49 7000 1.2363 0.7816
1.1975 15.53 7500 1.2442 0.7700
1.1907 16.56 8000 1.2569 0.7720
1.1231 17.6 8500 1.1386 0.7761
1.0873 18.63 9000 1.2105 0.7856
1.1242 19.67 9500 1.2142 0.7738
1.0367 20.7 10000 1.2121 0.7712
1.0869 21.74 10500 1.0782 0.7955
1.0353 22.77 11000 0.9918 0.8069
1.0324 23.81 11500 1.0908 0.7971
1.0145 24.84 12000 1.0945 0.7975
0.9951 25.88 12500 1.0005 0.8028
0.9483 26.92 13000 0.9638 0.8187
0.9304 27.95 13500 0.9761 0.8205
0.8835 28.99 14000 1.0620 0.8046
0.9097 30.02 14500 0.9138 0.8060
0.9293 31.06 15000 0.9180 0.8176
0.9043 32.09 15500 0.9215 0.8208
0.8581 33.13 16000 0.9625 0.8225
0.8638 34.16 16500 0.8586 0.8368
0.874 35.2 17000 1.0044 0.8135
0.8235 36.23 17500 0.9755 0.8184
0.8589 37.27 18000 0.9042 0.8292
0.8107 38.3 18500 0.8821 0.8272
0.8346 39.34 19000 0.9061 0.8248
0.8393 40.37 19500 0.9796 0.8235
0.789 41.41 20000 0.9015 0.8331
0.8121 42.44 20500 0.8589 0.8386
0.7709 43.48 21000 0.8836 0.8351
0.7922 44.51 21500 0.9524 0.8180
0.7457 45.55 22000 0.8350 0.8364
0.7386 46.58 22500 0.9025 0.8341
0.7515 47.62 23000 0.9092 0.8390
0.7324 48.65 23500 0.8322 0.8421
0.7314 49.69 24000 0.7968 0.8477
0.7442 50.72 24500 0.9305 0.8324
0.7074 51.76 25000 1.0011 0.8208
0.739 52.8 25500 0.8732 0.8331
0.7243 53.83 26000 0.7857 0.8480
0.6842 54.87 26500 0.7945 0.8377
0.6991 55.9 27000 0.9628 0.8275
0.6896 56.94 27500 0.8363 0.8410
0.6925 57.97 28000 0.8433 0.8392
0.7081 59.01 28500 1.0086 0.8223
0.6598 60.04 29000 0.9251 0.8333
0.6677 61.08 29500 0.8823 0.8437
0.695 62.11 30000 0.7751 0.8560
0.7108 63.15 30500 0.8452 0.8481
0.6721 64.18 31000 0.8560 0.8413
0.6571 65.22 31500 0.9800 0.8163
0.6891 66.25 32000 0.8106 0.8457
0.6541 67.29 32500 0.8197 0.8430
0.6559 68.32 33000 0.8678 0.8388
0.6554 69.36 33500 0.7396 0.8662
0.618 70.39 34000 0.8518 0.8376
0.6558 71.43 34500 0.7706 0.8409
0.6034 72.46 35000 0.7829 0.8518
0.6336 73.5 35500 0.7835 0.8591
0.6287 74.53 36000 0.7548 0.8575
0.6065 75.57 36500 0.8542 0.8508
0.6029 76.6 37000 0.8203 0.8405
0.6208 77.64 37500 0.7082 0.8661
0.64 78.67 38000 0.8505 0.8410
0.6144 79.71 38500 0.7246 0.8604
0.6507 80.75 39000 0.7150 0.8611
0.6177 81.78 39500 0.9332 0.84
0.6159 82.82 40000 0.6427 0.8733
0.5944 83.85 40500 0.7721 0.8411
0.6044 84.89 41000 0.8968 0.8449
0.6 85.92 41500 0.7673 0.8538
0.5899 86.96 42000 0.8039 0.8505
0.5812 87.99 42500 0.7467 0.8567
0.5977 89.03 43000 0.9534 0.8316
0.6019 90.06 43500 0.9170 0.8316
0.563 91.1 44000 0.7761 0.8569
0.6347 92.13 44500 0.7811 0.8577
0.5855 93.17 45000 0.7562 0.8606
0.6026 94.2 45500 0.7490 0.8636
0.5846 95.24 46000 0.7456 0.8487
0.5635 96.27 46500 0.8115 0.8495
0.5903 97.31 47000 0.8137 0.8448
0.576 98.34 47500 0.8441 0.8424
0.5745 99.38 48000 0.7266 0.8609
0.5915 100.41 48500 0.9169 0.8446
0.601 101.45 49000 0.7671 0.8576
0.5713 102.48 49500 0.7868 0.8487
0.5541 103.52 50000 0.7907 0.8569

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.0.0
  • Datasets 2.15.0
  • Tokenizers 0.15.0