--- license: apache-2.0 base_model: bert-base-multilingual-cased tags: - generated_from_trainer metrics: - accuracy model-index: - name: mt results: [] --- # mt This model is an adapter fine-tuned on top of of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on the Maltese ConceptNet dataset. It achieves the following results on the evaluation set: - Loss: 0.8117 - Accuracy: 0.8590 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - distributed_type: multi-GPU - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 50000 ### Training results | Training Loss | Epoch | Step | Validation Loss | Accuracy | |:-------------:|:------:|:-----:|:---------------:|:--------:| | 2.6689 | 1.04 | 500 | 2.1345 | 0.6677 | | 2.1415 | 2.07 | 1000 | 1.8436 | 0.6926 | | 1.9421 | 3.11 | 1500 | 1.7874 | 0.6907 | | 1.7588 | 4.14 | 2000 | 1.7605 | 0.7013 | | 1.6729 | 5.18 | 2500 | 1.7568 | 0.6957 | | 1.596 | 6.21 | 3000 | 1.5006 | 0.7273 | | 1.5778 | 7.25 | 3500 | 1.3924 | 0.7451 | | 1.4821 | 8.28 | 4000 | 1.6097 | 0.7099 | | 1.4183 | 9.32 | 4500 | 1.3552 | 0.7491 | | 1.4197 | 10.35 | 5000 | 1.2847 | 0.7513 | | 1.3156 | 11.39 | 5500 | 1.3173 | 0.7496 | | 1.2882 | 12.42 | 6000 | 1.2817 | 0.7738 | | 1.2692 | 13.46 | 6500 | 1.1892 | 0.7751 | | 1.2368 | 14.49 | 7000 | 1.2363 | 0.7816 | | 1.1975 | 15.53 | 7500 | 1.2442 | 0.7700 | | 1.1907 | 16.56 | 8000 | 1.2569 | 0.7720 | | 1.1231 | 17.6 | 8500 | 1.1386 | 0.7761 | | 1.0873 | 18.63 | 9000 | 1.2105 | 0.7856 | | 1.1242 | 19.67 | 9500 | 1.2142 | 0.7738 | | 1.0367 | 20.7 | 10000 | 1.2121 | 0.7712 | | 1.0869 | 21.74 | 10500 | 1.0782 | 0.7955 | | 1.0353 | 22.77 | 11000 | 0.9918 | 0.8069 | | 1.0324 | 23.81 | 11500 | 1.0908 | 0.7971 | | 1.0145 | 24.84 | 12000 | 1.0945 | 0.7975 | | 0.9951 | 25.88 | 12500 | 1.0005 | 0.8028 | | 0.9483 | 26.92 | 13000 | 0.9638 | 0.8187 | | 0.9304 | 27.95 | 13500 | 0.9761 | 0.8205 | | 0.8835 | 28.99 | 14000 | 1.0620 | 0.8046 | | 0.9097 | 30.02 | 14500 | 0.9138 | 0.8060 | | 0.9293 | 31.06 | 15000 | 0.9180 | 0.8176 | | 0.9043 | 32.09 | 15500 | 0.9215 | 0.8208 | | 0.8581 | 33.13 | 16000 | 0.9625 | 0.8225 | | 0.8638 | 34.16 | 16500 | 0.8586 | 0.8368 | | 0.874 | 35.2 | 17000 | 1.0044 | 0.8135 | | 0.8235 | 36.23 | 17500 | 0.9755 | 0.8184 | | 0.8589 | 37.27 | 18000 | 0.9042 | 0.8292 | | 0.8107 | 38.3 | 18500 | 0.8821 | 0.8272 | | 0.8346 | 39.34 | 19000 | 0.9061 | 0.8248 | | 0.8393 | 40.37 | 19500 | 0.9796 | 0.8235 | | 0.789 | 41.41 | 20000 | 0.9015 | 0.8331 | | 0.8121 | 42.44 | 20500 | 0.8589 | 0.8386 | | 0.7709 | 43.48 | 21000 | 0.8836 | 0.8351 | | 0.7922 | 44.51 | 21500 | 0.9524 | 0.8180 | | 0.7457 | 45.55 | 22000 | 0.8350 | 0.8364 | | 0.7386 | 46.58 | 22500 | 0.9025 | 0.8341 | | 0.7515 | 47.62 | 23000 | 0.9092 | 0.8390 | | 0.7324 | 48.65 | 23500 | 0.8322 | 0.8421 | | 0.7314 | 49.69 | 24000 | 0.7968 | 0.8477 | | 0.7442 | 50.72 | 24500 | 0.9305 | 0.8324 | | 0.7074 | 51.76 | 25000 | 1.0011 | 0.8208 | | 0.739 | 52.8 | 25500 | 0.8732 | 0.8331 | | 0.7243 | 53.83 | 26000 | 0.7857 | 0.8480 | | 0.6842 | 54.87 | 26500 | 0.7945 | 0.8377 | | 0.6991 | 55.9 | 27000 | 0.9628 | 0.8275 | | 0.6896 | 56.94 | 27500 | 0.8363 | 0.8410 | | 0.6925 | 57.97 | 28000 | 0.8433 | 0.8392 | | 0.7081 | 59.01 | 28500 | 1.0086 | 0.8223 | | 0.6598 | 60.04 | 29000 | 0.9251 | 0.8333 | | 0.6677 | 61.08 | 29500 | 0.8823 | 0.8437 | | 0.695 | 62.11 | 30000 | 0.7751 | 0.8560 | | 0.7108 | 63.15 | 30500 | 0.8452 | 0.8481 | | 0.6721 | 64.18 | 31000 | 0.8560 | 0.8413 | | 0.6571 | 65.22 | 31500 | 0.9800 | 0.8163 | | 0.6891 | 66.25 | 32000 | 0.8106 | 0.8457 | | 0.6541 | 67.29 | 32500 | 0.8197 | 0.8430 | | 0.6559 | 68.32 | 33000 | 0.8678 | 0.8388 | | 0.6554 | 69.36 | 33500 | 0.7396 | 0.8662 | | 0.618 | 70.39 | 34000 | 0.8518 | 0.8376 | | 0.6558 | 71.43 | 34500 | 0.7706 | 0.8409 | | 0.6034 | 72.46 | 35000 | 0.7829 | 0.8518 | | 0.6336 | 73.5 | 35500 | 0.7835 | 0.8591 | | 0.6287 | 74.53 | 36000 | 0.7548 | 0.8575 | | 0.6065 | 75.57 | 36500 | 0.8542 | 0.8508 | | 0.6029 | 76.6 | 37000 | 0.8203 | 0.8405 | | 0.6208 | 77.64 | 37500 | 0.7082 | 0.8661 | | 0.64 | 78.67 | 38000 | 0.8505 | 0.8410 | | 0.6144 | 79.71 | 38500 | 0.7246 | 0.8604 | | 0.6507 | 80.75 | 39000 | 0.7150 | 0.8611 | | 0.6177 | 81.78 | 39500 | 0.9332 | 0.84 | | 0.6159 | 82.82 | 40000 | 0.6427 | 0.8733 | | 0.5944 | 83.85 | 40500 | 0.7721 | 0.8411 | | 0.6044 | 84.89 | 41000 | 0.8968 | 0.8449 | | 0.6 | 85.92 | 41500 | 0.7673 | 0.8538 | | 0.5899 | 86.96 | 42000 | 0.8039 | 0.8505 | | 0.5812 | 87.99 | 42500 | 0.7467 | 0.8567 | | 0.5977 | 89.03 | 43000 | 0.9534 | 0.8316 | | 0.6019 | 90.06 | 43500 | 0.9170 | 0.8316 | | 0.563 | 91.1 | 44000 | 0.7761 | 0.8569 | | 0.6347 | 92.13 | 44500 | 0.7811 | 0.8577 | | 0.5855 | 93.17 | 45000 | 0.7562 | 0.8606 | | 0.6026 | 94.2 | 45500 | 0.7490 | 0.8636 | | 0.5846 | 95.24 | 46000 | 0.7456 | 0.8487 | | 0.5635 | 96.27 | 46500 | 0.8115 | 0.8495 | | 0.5903 | 97.31 | 47000 | 0.8137 | 0.8448 | | 0.576 | 98.34 | 47500 | 0.8441 | 0.8424 | | 0.5745 | 99.38 | 48000 | 0.7266 | 0.8609 | | 0.5915 | 100.41 | 48500 | 0.9169 | 0.8446 | | 0.601 | 101.45 | 49000 | 0.7671 | 0.8576 | | 0.5713 | 102.48 | 49500 | 0.7868 | 0.8487 | | 0.5541 | 103.52 | 50000 | 0.7907 | 0.8569 | ### Framework versions - Transformers 4.35.2 - Pytorch 2.0.0 - Datasets 2.15.0 - Tokenizers 0.15.0