Kushtrim/whisper-base-shqip-97h

Whisper Base Shqip

This model is a fine-tuned version of openai/whisper-base on the Audio Shqip 97 orë dataset. It achieves the following results on the evaluation set:

• Loss: 0.5274
• Wer: 40.1434

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.0357	0.3249	500	1.0437	70.9649
0.7862	0.6498	1000	0.7759	57.9971
0.6561	0.9747	1500	0.6805	51.6728
0.5704	1.2995	2000	0.6337	49.0896
0.5511	1.6244	2500	0.5968	47.4252
0.522	1.9493	3000	0.5740	47.2168
0.4252	2.2742	3500	0.5612	43.5865
0.4411	2.5991	4000	0.5487	43.2817
0.4434	2.9240	4500	0.5373	43.3737
0.3791	3.2489	5000	0.5353	42.3143
0.371	3.5737	5500	0.5297	41.3114
0.4173	3.8986	6000	0.5231	41.4012
0.3009	4.2235	6500	0.5276	40.9756
0.3337	4.5484	7000	0.5249	40.4393
0.3145	4.8733	7500	0.5222	40.2154
0.2897	5.1982	8000	0.5264	40.4925
0.2717	5.5231	8500	0.5256	40.6387
0.2947	5.8480	9000	0.5251	40.2753
0.2933	6.1728	9500	0.5268	40.5601
0.2644	6.4977	10000	0.5274	40.1434

Framework versions

• Transformers 4.45.2
• Pytorch 2.5.1+cu121
• Datasets 3.2.0
• Tokenizers 0.20.3