Hugging Face's logo

asr-africa
/
w2v-bert-2.0-lg-CV-Fleurs-5hrs-v10

Automatic Speech Recognition
Transformers
Safetensors
wav2vec2-bert
Generated from Trainer
Inference Endpoints
Model card Files Community
w2v-bert-2.0-lg-CV-Fleurs-5hrs-v10 / README.md
sulaimank's picture
sulaimank
End of training
f8c91e2 verified
preview code
|
raw
history blame contribute delete
3.61 kB
metadata
library_name: transformers
license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v-bert-2.0-lg-CV-Fleurs-5hrs-v10
    results: []

w2v-bert-2.0-lg-CV-Fleurs-5hrs-v10

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8414
  • Wer: 0.3891
  • Cer: 0.0838

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 8
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
1.6631 1.0 163 0.5729 0.5671 0.1312
0.4064 2.0 326 0.4681 0.4766 0.1051
0.3071 3.0 489 0.4348 0.4344 0.0961
0.2518 4.0 652 0.4442 0.4112 0.0885
0.2154 5.0 815 0.4503 0.4042 0.0877
0.1824 6.0 978 0.4146 0.4287 0.0910
0.1625 7.0 1141 0.4245 0.4082 0.0878
0.1354 8.0 1304 0.4579 0.4335 0.0881
0.1182 9.0 1467 0.4593 0.4242 0.0916
0.1025 10.0 1630 0.4587 0.4046 0.0881
0.0863 11.0 1793 0.5591 0.3991 0.0854
0.0723 12.0 1956 0.4954 0.4041 0.0863
0.0619 13.0 2119 0.5618 0.4127 0.0890
0.0543 14.0 2282 0.5675 0.4115 0.0892
0.0461 15.0 2445 0.6027 0.3968 0.0861
0.0412 16.0 2608 0.5939 0.4138 0.0895
0.0348 17.0 2771 0.6687 0.4157 0.0894
0.0342 18.0 2934 0.7066 0.3849 0.0838
0.0288 19.0 3097 0.7669 0.3899 0.0849
0.0233 20.0 3260 0.6945 0.4000 0.0865
0.0218 21.0 3423 0.7192 0.4086 0.0883
0.02 22.0 3586 0.6980 0.3940 0.0843
0.017 23.0 3749 0.7983 0.4014 0.0873
0.0153 24.0 3912 0.7599 0.3942 0.0853
0.0142 25.0 4075 0.7761 0.3993 0.0858
0.0137 26.0 4238 0.7491 0.3996 0.0857
0.0157 27.0 4401 0.7682 0.3994 0.0858
0.0113 28.0 4564 0.7784 0.4006 0.0875
0.0111 29.0 4727 0.8020 0.4020 0.0864
0.0105 30.0 4890 0.8414 0.3891 0.0838

Framework versions

  • Transformers 4.46.3
  • Pytorch 2.1.0+cu118
  • Datasets 3.1.0
  • Tokenizers 0.20.3