kiranpantha's picture
End of training
cb4658e verified
metadata
library_name: transformers
language:
  - ne
license: mit
base_model: kiranpantha/w2v-bert-2.0-nepali
tags:
  - generated_from_trainer
datasets:
  - kiranpantha/OpenSLR54-Balanced-Nepali
metrics:
  - wer
model-index:
  - name: Wave2Vec2-Bert2.0 - Kiran Pantha
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: OpenSLR54
          type: kiranpantha/OpenSLR54-Balanced-Nepali
          config: default
          split: test
          args: 'config: ne, split: train,test'
        metrics:
          - name: Wer
            type: wer
            value: 0.4301989457575242

Wave2Vec2-Bert2.0 - Kiran Pantha

This model is a fine-tuned version of kiranpantha/w2v-bert-2.0-nepali on the OpenSLR54 dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4052
  • Wer: 0.4302
  • Cer: 0.1029

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 2
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.7515 0.15 300 0.4814 0.4911 0.1183
0.6554 0.3 600 0.5699 0.5382 0.1385
0.6723 0.45 900 0.5463 0.5401 0.1395
0.6635 0.6 1200 0.5244 0.5043 0.1250
0.6132 0.75 1500 0.4725 0.4831 0.1184
0.5786 0.9 1800 0.4620 0.4702 0.1147
0.5639 1.05 2100 0.4810 0.4668 0.1140
0.4863 1.2 2400 0.4639 0.4766 0.1151
0.4784 1.35 2700 0.4527 0.4611 0.1108
0.456 1.5 3000 0.4229 0.4458 0.1089
0.4613 1.65 3300 0.4460 0.4478 0.1095
0.4506 1.8 3600 0.4166 0.4413 0.1047
0.4369 1.95 3900 0.4052 0.4302 0.1029

Framework versions

  • Transformers 4.45.0.dev0
  • Pytorch 2.4.1+cu121
  • Datasets 2.21.0
  • Tokenizers 0.19.1