---
base_model: demdecuong/vihealthbert-base-word
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: vihealthbert-w_mlm-ViMedNLI
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# vihealthbert-w_mlm-ViMedNLI

This model is a fine-tuned version of [demdecuong/vihealthbert-base-word](https://huggingface.co/demdecuong/vihealthbert-base-word) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1156
- Accuracy: 0.8341

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 19161
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- training_steps: 30000

### Training results

| Training Loss | Epoch    | Step  | Validation Loss | Accuracy |
|:-------------:|:--------:|:-----:|:---------------:|:--------:|
| 5.5327        | 10.5263  | 1000  | 2.7528          | 0.5890   |
| 1.9051        | 21.0526  | 2000  | 1.4678          | 0.7783   |
| 1.1194        | 31.5789  | 3000  | 1.1543          | 0.8020   |
| 0.831         | 42.1053  | 4000  | 1.0972          | 0.8147   |
| 0.6805        | 52.6316  | 5000  | 0.9968          | 0.8256   |
| 0.5937        | 63.1579  | 6000  | 1.0310          | 0.8243   |
| 0.5258        | 73.6842  | 7000  | 1.1045          | 0.8151   |
| 0.4569        | 84.2105  | 8000  | 1.0393          | 0.8254   |
| 0.4007        | 94.7368  | 9000  | 1.0684          | 0.8217   |
| 0.3632        | 105.2632 | 10000 | 1.1223          | 0.8182   |
| 0.3343        | 115.7895 | 11000 | 1.1048          | 0.8230   |
| 0.2998        | 126.3158 | 12000 | 1.0996          | 0.8218   |
| 0.2817        | 136.8421 | 13000 | 1.0880          | 0.8320   |
| 0.2568        | 147.3684 | 14000 | 1.1189          | 0.8216   |
| 0.2396        | 157.8947 | 15000 | 1.1026          | 0.8267   |
| 0.219         | 168.4211 | 16000 | 1.1284          | 0.8241   |
| 0.2028        | 178.9474 | 17000 | 1.1205          | 0.8243   |
| 0.1927        | 189.4737 | 18000 | 1.1104          | 0.8313   |
| 0.1841        | 200.0    | 19000 | 1.0284          | 0.8348   |
| 0.1687        | 210.5263 | 20000 | 1.1662          | 0.8266   |
| 0.1627        | 221.0526 | 21000 | 1.1330          | 0.8278   |
| 0.1564        | 231.5789 | 22000 | 1.1413          | 0.8265   |
| 0.1483        | 242.1053 | 23000 | 1.1836          | 0.8246   |
| 0.1439        | 252.6316 | 24000 | 1.2169          | 0.8179   |
| 0.1396        | 263.1579 | 25000 | 1.1871          | 0.8266   |
| 0.1364        | 273.6842 | 26000 | 1.1696          | 0.8301   |
| 0.1314        | 284.2105 | 27000 | 1.1557          | 0.8324   |
| 0.1295        | 294.7368 | 28000 | 1.1712          | 0.8298   |
| 0.1296        | 305.2632 | 29000 | 1.1821          | 0.8273   |
| 0.1251        | 315.7895 | 30000 | 1.1567          | 0.8262   |


### Framework versions

- Transformers 4.40.2
- Pytorch 2.2.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1