pashto-asr-v3

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1448
• Wer: 0.1396

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 1300
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.9698	0.8089	100	2.8928	0.9991
0.8095	1.6178	200	0.6035	0.4036
0.6152	2.4267	300	0.4857	0.3593
0.3951	3.2356	400	0.4661	0.3505
0.5493	4.0445	500	0.3651	0.2779
0.4588	4.8534	600	0.3244	0.2632
0.3616	5.6623	700	0.2954	0.2490
0.1938	6.4712	800	0.2655	0.2341
0.2047	7.2801	900	0.2510	0.2022
0.2596	8.0890	1000	0.1953	0.1756
0.1871	8.8979	1100	0.1716	0.1642
0.0768	9.7068	1200	0.1559	0.1554
0.1021	10.5157	1300	0.1448	0.1396

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1