Hugging Face's logo

kiranpantha
/
w2v-bert-2.0-nepali

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
Nepali
wav2vec2-bert
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Metrics Community
w2v-bert-2.0-nepali / README.md
kiranpantha's picture
kiranpantha
End of training
1cc0b16 verified
preview code
|
raw
history blame contribute delete
No virus
3.4 kB
metadata
library_name: transformers
language:
  - ne
license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - generated_from_trainer
datasets:
  - kiranpantha/OpenSLR54-Balanced-Nepali
metrics:
  - wer
model-index:
  - name: Wave2Vec2-Bert2.0 - Kiran Pantha
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: OpenSLR54
          type: kiranpantha/OpenSLR54-Balanced-Nepali
          config: default
          split: test
          args: 'config: ne, split: train,test'
        metrics:
          - name: Wer
            type: wer
            value: 0.25254629629629627

Wave2Vec2-Bert2.0 - Kiran Pantha

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the OpenSLR54 dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2212
  • Wer: 0.2525
  • Cer: 0.0565

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 2
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.4436 0.0900 300 0.5638 0.5560 0.1447
0.5495 0.1800 600 0.6876 0.6171 0.1641
0.6148 0.2699 900 0.6872 0.6211 0.1724
0.564 0.3599 1200 0.5503 0.5162 0.1326
0.4964 0.4499 1500 0.5831 0.5319 0.1318
0.4437 0.5399 1800 0.4913 0.4935 0.1202
0.4441 0.6299 2100 0.4754 0.4764 0.1193
0.3861 0.7199 2400 0.4357 0.4361 0.1055
0.3811 0.8098 2700 0.4282 0.4137 0.0976
0.3754 0.8998 3000 0.3905 0.4069 0.0975
0.3511 0.9898 3300 0.3547 0.3692 0.0863
0.2496 1.0798 3600 0.3297 0.3433 0.0796
0.242 1.1698 3900 0.3125 0.3315 0.0770
0.2378 1.2597 4200 0.3158 0.3336 0.0757
0.2274 1.3497 4500 0.2871 0.3097 0.0722
0.2142 1.4397 4800 0.3010 0.3058 0.0712
0.1949 1.5297 5100 0.2767 0.2944 0.0678
0.198 1.6197 5400 0.2487 0.2824 0.0639
0.1806 1.7097 5700 0.2376 0.2674 0.0612
0.1675 1.7996 6000 0.2293 0.2630 0.0595
0.1671 1.8896 6300 0.2248 0.2581 0.0576
0.1526 1.9796 6600 0.2212 0.2525 0.0565

Framework versions

  • Transformers 4.45.0.dev0
  • Pytorch 2.4.1+cu121
  • Datasets 2.21.0
  • Tokenizers 0.19.1