tgrhn/wav2vec2-turkish-7

Wav2Vec2-XLS-TR

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the Common Voice 17 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2925

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.6208	1.0	2901	0.3714
0.4852	2.0	5802	0.3358
0.4807	3.0	8703	0.4309
0.441	4.0	11604	0.4345
0.3965	5.0	14505	0.3815
0.3429	6.0	17406	0.3601
0.3074	7.0	20307	0.3551
0.2913	8.0	23208	0.3477
0.2744	9.0	26109	0.3262
0.2611	10.0	29010	0.3225
0.2359	11.0	31911	0.3244
0.2398	12.0	34812	0.3053
0.2085	13.0	37713	0.3058
0.1903	14.0	40614	0.3219
0.1827	15.0	43515	0.2911
0.1654	16.0	46416	0.2890
0.1548	17.0	49317	0.2945
0.1467	18.0	52218	0.2807
0.1399	19.0	55119	0.2914
0.1272	20.0	58020	0.2974
0.1162	21.0	60921	0.2991
0.1056	22.0	63822	0.2743
0.1004	23.0	66723	0.2932
0.0926	24.0	69624	0.2806
0.0855	25.0	72525	0.2872
0.0802	26.0	75426	0.2732
0.078	27.0	78327	0.2844
0.0689	28.0	81228	0.2884
0.0634	29.0	84129	0.2925
0.0616	30.0	87030	0.2925

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1