Mihaj's picture
Upload tokenizer
9b9994e verified
metadata
base_model: NbAiLab/nb-whisper-small-verbatim
license: apache-2.0
metrics:
  - wer
tags:
  - generated_from_trainer
model-index:
  - name: nb-whisper-small-karelian-CodeSwitching
    results: []

nb-whisper-small-karelian-CodeSwitching

This model is a fine-tuned version of NbAiLab/nb-whisper-small-verbatim on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7241
  • Wer: 0.3278
  • Cer: 0.0999

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • distributed_type: multi-GPU
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • training_steps: 10000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.1848 1.1338 500 0.6758 0.4066 0.1227
0.0796 2.2676 1000 0.6685 0.3549 0.1015
0.0492 3.4014 1500 0.7090 0.3758 0.1088
0.0426 4.5351 2000 0.7095 0.3704 0.1083
0.0311 5.6689 2500 0.7148 0.3742 0.1138
0.027 6.8027 3000 0.7173 0.3454 0.1004
0.0201 7.9365 3500 0.7261 0.3813 0.1325
0.0165 9.0703 4000 0.7158 0.3417 0.0982
0.0179 10.2041 4500 0.7261 0.3495 0.1036
0.0101 11.3379 5000 0.7275 0.3315 0.0978
0.0086 12.4717 5500 0.7437 0.3400 0.1081
0.0058 13.6054 6000 0.7524 0.3410 0.1026
0.0056 14.7392 6500 0.7256 0.3407 0.1015
0.0075 15.8730 7000 0.7202 0.3312 0.0987
0.0047 17.0068 7500 0.7266 0.3359 0.1025
0.0046 18.1406 8000 0.7271 0.3312 0.0973
0.0039 19.2744 8500 0.7334 0.3353 0.0999
0.0025 20.4082 9000 0.7280 0.3295 0.0987
0.0022 21.5420 9500 0.7290 0.3254 0.0972
0.0031 22.6757 10000 0.7241 0.3278 0.0999

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.1+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1