---
base_model: facebook/w2v-bert-2.0
license: mit
metrics:
- wer
tags:
- generated_from_trainer
model-index:
- name: malayalam_combined_
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/krishnan-aravind/huggingface/runs/7l3na48x)
# malayalam_combined_

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5025
- Wer: 0.4256

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 50
- num_epochs: 25
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Wer    |
|:-------------:|:-------:|:-----:|:---------------:|:------:|
| 0.8238        | 0.2031  | 500   | 0.8281          | 0.6745 |
| 0.7415        | 0.4063  | 1000  | 0.7477          | 0.6446 |
| 0.6913        | 0.6094  | 1500  | 0.6962          | 0.6072 |
| 0.6401        | 0.8125  | 2000  | 0.6981          | 0.5929 |
| 0.5864        | 1.0156  | 2500  | 0.6809          | 0.5712 |
| 0.5843        | 1.2188  | 3000  | 0.6125          | 0.5691 |
| 0.5547        | 1.4219  | 3500  | 0.6110          | 0.5616 |
| 0.5657        | 1.6250  | 4000  | 0.5882          | 0.5464 |
| 0.5809        | 1.8282  | 4500  | 0.5776          | 0.5481 |
| 0.5464        | 2.0313  | 5000  | 0.5689          | 0.5278 |
| 0.4974        | 2.2344  | 5500  | 0.5926          | 0.5428 |
| 0.5012        | 2.4375  | 6000  | 0.5622          | 0.5384 |
| 0.5162        | 2.6407  | 6500  | 0.5697          | 0.5179 |
| 0.5006        | 2.8438  | 7000  | 0.5357          | 0.5375 |
| 0.4661        | 3.0469  | 7500  | 0.5255          | 0.5255 |
| 0.4658        | 3.2501  | 8000  | 0.5182          | 0.5002 |
| 0.4716        | 3.4532  | 8500  | 0.5176          | 0.5044 |
| 0.4658        | 3.6563  | 9000  | 0.5139          | 0.5061 |
| 0.5031        | 3.8594  | 9500  | 0.5114          | 0.5068 |
| 0.4482        | 4.0626  | 10000 | 0.5331          | 0.5101 |
| 0.4678        | 4.2657  | 10500 | 0.5165          | 0.5126 |
| 0.4353        | 4.4688  | 11000 | 0.5292          | 0.5112 |
| 0.4711        | 4.6719  | 11500 | 0.5178          | 0.4979 |
| 0.4574        | 4.8751  | 12000 | 0.5215          | 0.5100 |
| 0.4246        | 5.0782  | 12500 | 0.5190          | 0.4938 |
| 0.4164        | 5.2813  | 13000 | 0.5504          | 0.4898 |
| 0.4181        | 5.4845  | 13500 | 0.5045          | 0.4979 |
| 0.4279        | 5.6876  | 14000 | 0.5118          | 0.4932 |
| 0.4244        | 5.8907  | 14500 | 0.4970          | 0.4842 |
| 0.4038        | 6.0938  | 15000 | 0.5013          | 0.4776 |
| 0.4179        | 6.2970  | 15500 | 0.5061          | 0.4762 |
| 0.3812        | 6.5001  | 16000 | 0.4987          | 0.4689 |
| 0.4217        | 6.7032  | 16500 | 0.4986          | 0.4807 |
| 0.3989        | 6.9064  | 17000 | 0.4905          | 0.4709 |
| 0.3741        | 7.1095  | 17500 | 0.4842          | 0.4700 |
| 0.3743        | 7.3126  | 18000 | 0.4869          | 0.4734 |
| 0.3785        | 7.5157  | 18500 | 0.4692          | 0.4690 |
| 0.3759        | 7.7189  | 19000 | 0.4691          | 0.4646 |
| 0.3809        | 7.9220  | 19500 | 0.4736          | 0.4720 |
| 0.3499        | 8.1251  | 20000 | 0.4787          | 0.4691 |
| 0.3523        | 8.3283  | 20500 | 0.4689          | 0.4680 |
| 0.3551        | 8.5314  | 21000 | 0.4792          | 0.4567 |
| 0.3672        | 8.7345  | 21500 | 0.4760          | 0.4652 |
| 0.3554        | 8.9376  | 22000 | 0.4649          | 0.4648 |
| 0.3182        | 9.1408  | 22500 | 0.4853          | 0.4565 |
| 0.3412        | 9.3439  | 23000 | 0.4958          | 0.4616 |
| 0.3494        | 9.5470  | 23500 | 0.4971          | 0.4527 |
| 0.3426        | 9.7502  | 24000 | 0.4959          | 0.4554 |
| 0.3365        | 9.9533  | 24500 | 0.4659          | 0.4582 |
| 0.3179        | 10.1564 | 25000 | 0.4807          | 0.4445 |
| 0.3361        | 10.3595 | 25500 | 0.4700          | 0.4535 |
| 0.3234        | 10.5627 | 26000 | 0.4562          | 0.4542 |
| 0.3296        | 10.7658 | 26500 | 0.4682          | 0.4452 |
| 0.3148        | 10.9689 | 27000 | 0.4716          | 0.4521 |
| 0.3112        | 11.1720 | 27500 | 0.4537          | 0.4473 |
| 0.3246        | 11.3752 | 28000 | 0.4594          | 0.4444 |
| 0.3062        | 11.5783 | 28500 | 0.4544          | 0.4445 |
| 0.2979        | 11.7814 | 29000 | 0.4531          | 0.4516 |
| 0.3108        | 11.9846 | 29500 | 0.4514          | 0.4428 |
| 0.2876        | 12.1877 | 30000 | 0.4598          | 0.4402 |
| 0.2911        | 12.3908 | 30500 | 0.4554          | 0.4426 |
| 0.2963        | 12.5939 | 31000 | 0.4641          | 0.4483 |
| 0.296         | 12.7971 | 31500 | 0.4575          | 0.4394 |
| 0.2777        | 13.0002 | 32000 | 0.4586          | 0.4444 |
| 0.2782        | 13.2033 | 32500 | 0.4498          | 0.4461 |
| 0.2695        | 13.4065 | 33000 | 0.4696          | 0.4450 |
| 0.286         | 13.6096 | 33500 | 0.4630          | 0.4383 |
| 0.279         | 13.8127 | 34000 | 0.4618          | 0.4401 |
| 0.2584        | 14.0158 | 34500 | 0.4526          | 0.4356 |
| 0.267         | 14.2190 | 35000 | 0.4726          | 0.4297 |
| 0.2667        | 14.4221 | 35500 | 0.4572          | 0.4308 |
| 0.2592        | 14.6252 | 36000 | 0.4795          | 0.4325 |
| 0.2592        | 14.8284 | 36500 | 0.4528          | 0.4303 |
| 0.2644        | 15.0315 | 37000 | 0.4604          | 0.4306 |
| 0.2312        | 15.2346 | 37500 | 0.4632          | 0.4367 |
| 0.2408        | 15.4377 | 38000 | 0.4670          | 0.4324 |
| 0.2489        | 15.6409 | 38500 | 0.4580          | 0.4253 |
| 0.2652        | 15.8440 | 39000 | 0.4581          | 0.4375 |
| 0.2367        | 16.0471 | 39500 | 0.4770          | 0.4213 |
| 0.2366        | 16.2503 | 40000 | 0.4751          | 0.4243 |
| 0.2267        | 16.4534 | 40500 | 0.4622          | 0.4282 |
| 0.2461        | 16.6565 | 41000 | 0.4671          | 0.4249 |
| 0.2326        | 16.8596 | 41500 | 0.4736          | 0.4293 |
| 0.2121        | 17.0628 | 42000 | 0.4905          | 0.4300 |
| 0.222         | 17.2659 | 42500 | 0.4782          | 0.4261 |
| 0.2202        | 17.4690 | 43000 | 0.4670          | 0.4250 |
| 0.2141        | 17.6722 | 43500 | 0.4688          | 0.4259 |
| 0.2231        | 17.8753 | 44000 | 0.4718          | 0.4254 |
| 0.2144        | 18.0784 | 44500 | 0.5025          | 0.4256 |


### Framework versions

- Transformers 4.43.0.dev0
- Pytorch 1.14.0a0+44dac51
- Datasets 2.16.1
- Tokenizers 0.19.1