File size: 3,493 Bytes
84f0718 bdcb1fc 84f0718 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
---
license: apache-2.0
tags:
- generated_from_keras_callback
base_model: bert-base-multilingual-cased
model-index:
- name: xmelus/mbert
results: []
---
This is a model card copied from original Tensorflow model version: https://huggingface.co/fimu-docproc-research/mbert-finetuned
# xmelus/mbert
This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 1.5424
- Train Accuracy: 0.1446
- Validation Loss: 1.5269
- Validation Accuracy: 0.1461
- Finished epochs: 24
### Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-05, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': -596, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 1000, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: mixed_float16
### Training results
Epoch 1/50
loss: 2.9925 - accuracy: 0.1059 - val_loss: 1.9812 - val_accuracy: 0.1331
Epoch 2/50
loss: 1.9979 - accuracy: 0.1307 - val_loss: 1.6063 - val_accuracy: 0.1429
Epoch 3/50
loss: 1.5798 - accuracy: 0.1434 - val_loss: 1.5332 - val_accuracy: 0.1461
Epoch 4/50
loss: 1.5325 - accuracy: 0.1451 - val_loss: 1.5285 - val_accuracy: 0.1458
Epoch 5/50
loss: 1.5415 - accuracy: 0.1448 - val_loss: 1.5449 - val_accuracy: 0.1457
Epoch 6/50
loss: 1.5395 - accuracy: 0.1448 - val_loss: 1.5448 - val_accuracy: 0.1456
Epoch 7/50
loss: 1.5463 - accuracy: 0.1446 - val_loss: 1.5421 - val_accuracy: 0.1454
Epoch 8/50
loss: 1.5352 - accuracy: 0.1451 - val_loss: 1.5536 - val_accuracy: 0.1453
Epoch 9/50
oss: 1.5230 - accuracy: 0.1451 - val_loss: 1.5097 - val_accuracy: 0.1466
Epoch 10/50
loss: 1.5318 - accuracy: 0.1449 - val_loss: 1.5303 - val_accuracy: 0.1460
Epoch 11/50
loss: 1.5364 - accuracy: 0.1448 - val_loss: 1.5280 - val_accuracy: 0.1462
Epoch 12/50
loss: 1.5411 - accuracy: 0.1444 - val_loss: 1.5493 - val_accuracy: 0.1455
Epoch 13/50
loss: 1.5378 - accuracy: 0.1446 - val_loss: 1.5473 - val_accuracy: 0.1456
Epoch 14/50
loss: 1.5357 - accuracy: 0.1449 - val_loss: 1.5310 - val_accuracy: 0.1457
Epoch 15/50
loss: 1.5424 - accuracy: 0.1446 - val_loss: 1.5269 - val_accuracy: 0.1461
Epoch 16/50
loss: 1.5314 - accuracy: 0.1450 - val_loss: 1.5392 - val_accuracy: 0.1456
Epoch 17/50
loss: 1.5309 - accuracy: 0.1451 - val_loss: 1.5567 - val_accuracy: 0.1454
Epoch 18/50
loss: 1.5279 - accuracy: 0.1450 - val_loss: 1.5561 - val_accuracy: 0.1452
Epoch 19/50
loss: 1.5311 - accuracy: 0.1450 - val_loss: 1.5400 - val_accuracy: 0.1460
Epoch 20/50
loss: 1.5332 - accuracy: 0.1449 - val_loss: 1.5347 - val_accuracy: 0.1460
Epoch 21/50
loss: 1.5319 - accuracy: 0.1452 - val_loss: 1.5410 - val_accuracy: 0.1458
Epoch 22/50
loss: 1.5327 - accuracy: 0.1449 - val_loss: 1.5352 - val_accuracy: 0.1460
Epoch 23/50
loss: 1.5278 - accuracy: 0.1451 - val_loss: 1.5289 - val_accuracy: 0.1458
Epoch 24/50
loss: 1.5234 - accuracy: 0.1451 - val_loss: 1.5568 - val_accuracy: 0.1449
### Framework versions
- Transformers 4.22.1
- Torch 1.13.1
- Datasets 2.5.1
- Tokenizers 0.12.1
|