malayalam_combined_Read

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1389
• Wer: 0.1305

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.3294	0.4378	500	0.2872	0.3888
0.2522	0.8757	1000	0.2404	0.3214
0.2017	1.3135	1500	0.2100	0.3247
0.19	1.7513	2000	0.2004	0.2744
0.1443	2.1891	2500	0.1791	0.2370
0.1484	2.6270	3000	0.1684	0.2309
0.1232	3.0648	3500	0.1672	0.2114
0.1135	3.5026	4000	0.1584	0.2031
0.1152	3.9405	4500	0.1457	0.1841
0.0904	4.3783	5000	0.1528	0.1870
0.0887	4.8161	5500	0.1455	0.1726
0.0768	5.2539	6000	0.1471	0.1636
0.069	5.6918	6500	0.1463	0.1578
0.0587	6.1296	7000	0.1401	0.1582
0.0537	6.5674	7500	0.1377	0.1485
0.0534	7.0053	8000	0.1278	0.1459
0.0397	7.4431	8500	0.1284	0.1418
0.0433	7.8809	9000	0.1274	0.1414
0.0323	8.3187	9500	0.1299	0.1357
0.0364	8.7566	10000	0.1340	0.1348
0.0219	9.1944	10500	0.1350	0.1335
0.0245	9.6322	11000	0.1389	0.1305

Framework versions

• Transformers 4.43.0.dev0
• Pytorch 1.14.0a0+44dac51
• Datasets 2.16.1
• Tokenizers 0.19.1