metadata
language:
- 'no'
license: apache-2.0
tags:
- audio
- asr
- automatic-speech-recognition
- hf-asr-leaderboard
model-index:
- name: nb-whisper-small-publicbeta-25k
results: []
nb-whisper-small-publicbeta-25k
This model is a fine-tuned version of openai/whisper-small on the NbAiLab/ncc_speech2 dataset.
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- lr_scheduler_type: linear
- per_device_train_batch_size: 32
- total_train_batch_size_per_node: 128
- total_train_batch_size: 1024
- total_optimization_steps: 25,000
- starting_optimization_step: None
- finishing_optimization_step: 25,000
- num_train_dataset_workers: 32
- num_hosts: 8
- total_num_training_examples: 25,600,000
- steps_per_epoch: 7313
- num_beams: 5
- weight_decay: 0.01
- adam_beta1: 0.9
- adam_beta2: 0.98
- adam_epsilon: 1e-06
- dropout: True
- bpe_dropout_probability: 0.1
- activation_dropout_probability: 0.1
Training results
| step | validation_fleurs_loss | train_loss | validation_fleurs_wer | validation_fleurs_cer | validation_fleurs_exact_wer | validation_fleurs_exact_cer | validation_stortinget_loss | validation_stortinget_wer | validation_stortinget_cer | validation_stortinget_exact_wer | validation_stortinget_exact_cer |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1.2013 | 3.1115 | 218.8876 | 174.4279 | 388.7694 | 278.9901 | 1.4191 | 71.3727 | 46.4810 | 76.7531 | 49.0057 |
| 1000 | 0.5627 | 1.1938 | 16.3593 | 6.2586 | 20.0717 | 7.2820 | 0.4640 | 20.7725 | 11.8840 | 24.4401 | 12.5992 |
| 2000 | 0.3961 | 0.9944 | 11.7192 | 4.0146 | 15.4719 | 4.9384 | 0.3737 | 16.5674 | 10.1748 | 20.0976 | 10.8109 |
| 3000 | 0.3696 | 0.9185 | 10.8269 | 4.1576 | 14.7551 | 5.1220 | 0.3426 | 14.9167 | 9.5103 | 18.3471 | 10.1061 |
| 4000 | 0.3467 | 0.8298 | 9.7858 | 4.2513 | 13.6201 | 5.1558 | 0.3251 | 14.3438 | 9.2267 | 17.7666 | 9.8219 |
| 5000 | 0.3266 | 0.8400 | 10.0833 | 4.2711 | 13.8889 | 5.2138 | 0.3110 | 13.9022 | 9.1039 | 17.2299 | 9.6697 |
| 6000 | 0.3280 | 0.7875 | 8.7745 | 3.3636 | 12.6344 | 4.3295 | 0.3058 | 13.5598 | 8.8853 | 16.9561 | 9.4543 |
| 7000 | 0.3177 | 0.7937 | 8.5961 | 3.7581 | 12.7539 | 4.6775 | 0.2991 | 13.1425 | 8.6226 | 16.4905 | 9.1878 |
| 8000 | 0.3383 | 0.7872 | 8.8935 | 3.8666 | 12.9630 | 4.7934 | 0.2917 | 13.0831 | 8.6552 | 16.4486 | 9.2255 |
| 9000 | 0.3320 | 0.7526 | 9.1612 | 4.0738 | 13.0526 | 5.0495 | 0.2899 | 12.8380 | 8.4996 | 16.1350 | 9.0495 |
| 10000 | 0.3267 | 0.7547 | 9.5181 | 4.1280 | 13.3513 | 5.1462 | 0.2894 | 12.7106 | 8.4593 | 16.0502 | 9.0189 |
| 11000 | 0.3358 | 0.7120 | 9.0125 | 4.1379 | 13.4409 | 5.1703 | 0.2889 | 12.8828 | 8.5885 | 16.1915 | 9.1459 |
| 12000 | 0.3179 | 0.7387 | 9.1910 | 4.2563 | 13.5006 | 5.2331 | 0.2825 | 12.6795 | 8.4383 | 16.0152 | 8.9950 |
| 13000 | 0.3152 | 0.7295 | 8.7448 | 4.0541 | 12.7539 | 4.9529 | 0.2832 | 12.5267 | 8.4567 | 15.8700 | 9.0105 |
Framework versions
- Transformers 4.31.0.dev0
- Datasets 2.13.0
- Tokenizers 0.13.3