wav2vec2-large-xls-r-1b-frisian-cv-8-10h
This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the common_voice_8_0 dataset. It achieves the following results on the evaluation set:
- Loss: 0.1207
- Wer: 0.0961
And on the test set:
- Wer: 0.0883
Model description
This model has been developed for my Master's thesis in "Voice Technology" at Rijksuniversiteit Groningen - Campus Fryslân. It corresponds to experiment 3 where I use as training set 10 hours of Frisian speech randomly selected from all validated data except the test and evaluation sets.
Intended uses & limitations
The intended use is for recognizing Frisian speech.
Limitations include no LM rescoring and using version 8.0 of Common Voice instead of 13.0.
Training and evaluation data
The evaluation split used is the one available in the Common Voice 8.0 Frisian subset. The train split is 10 hours of Frisian randomly selected from validated data except for the recordings from test and evaluation splits.
Training procedure
The script used for training this model can be found in this GitHub repository: link.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Wer |
---|---|---|---|---|
5.6342 | 1.32 | 300 | 2.9760 | 1.0 |
2.2716 | 2.63 | 600 | 0.6877 | 0.6024 |
1.1303 | 3.95 | 900 | 0.3522 | 0.3450 |
0.9038 | 5.26 | 1200 | 0.2714 | 0.2603 |
0.846 | 6.58 | 1500 | 0.2143 | 0.2036 |
0.8044 | 7.89 | 1800 | 0.1829 | 0.1788 |
0.7069 | 9.21 | 2100 | 0.1751 | 0.1667 |
0.6995 | 10.53 | 2400 | 0.1741 | 0.1727 |
0.7115 | 11.84 | 2700 | 0.1591 | 0.1486 |
0.677 | 13.16 | 3000 | 0.1636 | 0.1459 |
0.6032 | 14.47 | 3300 | 0.1535 | 0.1439 |
0.6218 | 15.79 | 3600 | 0.1427 | 0.1406 |
0.6519 | 17.11 | 3900 | 0.1498 | 0.1488 |
0.5739 | 18.42 | 4200 | 0.1438 | 0.1319 |
0.567 | 19.74 | 4500 | 0.1379 | 0.1322 |
0.4982 | 21.05 | 4800 | 0.1315 | 0.1237 |
0.5825 | 22.37 | 5100 | 0.1349 | 0.1252 |
0.5085 | 23.68 | 5400 | 0.1297 | 0.1233 |
0.4946 | 25.0 | 5700 | 0.1343 | 0.1127 |
0.5677 | 26.32 | 6000 | 0.1323 | 0.1228 |
0.4858 | 27.63 | 6300 | 0.1292 | 0.1098 |
0.4709 | 28.95 | 6600 | 0.1267 | 0.1204 |
0.3241 | 30.26 | 6900 | 0.1315 | 0.1274 |
0.2796 | 31.58 | 7200 | 0.1315 | 0.1202 |
0.3171 | 32.89 | 7500 | 0.1315 | 0.1200 |
0.2591 | 34.21 | 7800 | 0.1322 | 0.1106 |
0.2716 | 35.53 | 8100 | 0.1233 | 0.1030 |
0.2446 | 36.84 | 8400 | 0.1273 | 0.1087 |
0.2377 | 38.16 | 8700 | 0.1243 | 0.1101 |
0.2183 | 39.47 | 9000 | 0.1230 | 0.1116 |
0.2059 | 40.79 | 9300 | 0.1240 | 0.1001 |
0.1916 | 42.11 | 9600 | 0.1223 | 0.1003 |
0.196 | 43.42 | 9900 | 0.1246 | 0.0965 |
0.1969 | 44.74 | 10200 | 0.1222 | 0.1038 |
0.1951 | 46.05 | 10500 | 0.1208 | 0.1003 |
0.1809 | 47.37 | 10800 | 0.1213 | 0.1003 |
0.1793 | 48.68 | 11100 | 0.1202 | 0.0959 |
0.1837 | 50.0 | 11400 | 0.1207 | 0.0961 |
Framework versions
- Transformers 4.28.1
- Pytorch 2.0.0+cu117
- Datasets 2.11.0
- Tokenizers 0.13.3
- Downloads last month
- 3
Evaluation results
- Wer on common_voice_8_0validation set self-reported0.096
- Wer on common_voice_8_0test set self-reported0.088