wav2vec2-mms-1b-malayalam-colab-CV17.0

This model is a fine-tuned version of facebook/mms-1b-all on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 4.7578
• Wer: 1.0316
• Cer: 0.7469
• Bleu: {'bleu': 0.0, 'precisions': [0.0008639308855291577, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7118010694449419, 'length_ratio': 0.7462927143778207, 'translation_length': 2315, 'reference_length': 3102}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.15
• training_steps: 2000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer	Bleu
21.8941	3.1496	200	14.1370	1.0	1.0292	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.08102287291060646, 'length_ratio': 0.2846550612508059, 'translation_length': 883, 'reference_length': 3102}
8.5677	6.2992	400	6.3226	1.0071	0.8568	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.2500351312096836, 'length_ratio': 0.4190844616376531, 'translation_length': 1300, 'reference_length': 3102}
6.6037	9.4488	600	5.6520	1.0351	0.7789	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.6082895680797644, 'length_ratio': 0.6679561573178594, 'translation_length': 2072, 'reference_length': 3102}
6.0799	12.5984	800	5.2294	1.0574	0.7594	{'bleu': 0.0, 'precisions': [0.00041631973355537054, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7471989379147929, 'length_ratio': 0.7743391360412637, 'translation_length': 2402, 'reference_length': 3102}
5.7804	15.7480	1000	5.1733	1.0329	0.7629	{'bleu': 0.0, 'precisions': [0.0, 0.0, 0.0, 0.0], 'brevity_penalty': 0.6734028056038747, 'length_ratio': 0.7166344294003868, 'translation_length': 2223, 'reference_length': 3102}
5.4453	18.8976	1200	4.9821	1.0525	0.7467	{'bleu': 0.0, 'precisions': [0.0004042037186742118, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7758159728382187, 'length_ratio': 0.7975499677627337, 'translation_length': 2474, 'reference_length': 3102}
5.2983	22.0472	1400	4.8784	1.0297	0.7521	{'bleu': 0.0, 'precisions': [0.0008806693086745927, 0.0, 0.0, 0.0], 'brevity_penalty': 0.693559602972643, 'length_ratio': 0.7321083172147002, 'translation_length': 2271, 'reference_length': 3102}
5.0862	25.1969	1600	4.7948	1.0358	0.7446	{'bleu': 0.0, 'precisions': [0.0008399832003359933, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7387365301547462, 'length_ratio': 0.7675693101225016, 'translation_length': 2381, 'reference_length': 3102}
4.9884	28.3465	1800	4.7578	1.0316	0.7479	{'bleu': 0.0, 'precisions': [0.000864304235090752, 0.0, 0.0, 0.0], 'brevity_penalty': 0.711389009553914, 'length_ratio': 0.7459703417150225, 'translation_length': 2314, 'reference_length': 3102}
5.0447	31.4961	2000	4.7578	1.0316	0.7469	{'bleu': 0.0, 'precisions': [0.0008639308855291577, 0.0, 0.0, 0.0], 'brevity_penalty': 0.7118010694449419, 'length_ratio': 0.7462927143778207, 'translation_length': 2315, 'reference_length': 3102}

Framework versions

• Transformers 4.42.4
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1