---
tags:
- generated_from_trainer
datasets:
- common_voice_13_0
metrics:
- wer
model-index:
- name: DeCRED_small_cv
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# DeCRED_small_cv

This model is a fine-tuned version of [](https://huggingface.co/) on the common_voice_13_0 dataset.
It achieves the following results on the evaluation set:
- Cer: 0.0855
- Deletions: 4133
- Hits: 123172
- Insertions: 3671
- Loss: 1.1174
- Mer: 0.1889
- Substitutions: 20876
- Wer: 0.1935
- Wil: 0.3069
- Wip: 0.6931

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.002
- train_batch_size: 256
- eval_batch_size: 128
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- total_train_batch_size: 512
- total_eval_batch_size: 256
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 10000
- num_epochs: 20.0

### Training results

| Training Loss | Epoch | Step  | Cer    | Deletions | Hits   | Insertions | Validation Loss | Mer    | Substitutions | Wer    | Wil    | Wip    |
|:-------------:|:-----:|:-----:|:------:|:---------:|:------:|:----------:|:---------------:|:------:|:-------------:|:------:|:------:|:------:|
| 1.3459        | 5.0   | 7445  | 0.1676 | 7277      | 102705 | 6830       | 1.5193          | 0.3374 | 38199         | 0.3530 | 0.5182 | 0.4818 |
| 1.3006        | 6.0   | 8934  | 0.1584 | 6381      | 105485 | 7074       | 1.4779          | 0.3206 | 36315         | 0.3359 | 0.4956 | 0.5044 |
| 1.7549        | 7.0   | 10423 | 0.1620 | 6967      | 104307 | 6557       | 1.4874          | 0.3259 | 36907         | 0.3403 | 0.5031 | 0.4969 |
| 1.5621        | 8.0   | 11912 | 0.1361 | 5203      | 111002 | 6489       | 1.3687          | 0.2823 | 31976         | 0.2947 | 0.4437 | 0.5563 |
| 1.4988        | 9.0   | 13401 | 0.1298 | 5025      | 112303 | 6136       | 1.3308          | 0.2723 | 30853         | 0.2835 | 0.4299 | 0.5701 |
| 1.4495        | 10.0  | 14890 | 0.1212 | 5505      | 113957 | 5063       | 1.2901          | 0.2564 | 28719         | 0.2651 | 0.4068 | 0.5932 |
| 1.4092        | 11.0  | 16379 | 0.1158 | 5162      | 115248 | 4910       | 1.2625          | 0.2472 | 27771         | 0.2554 | 0.3941 | 0.6059 |
| 1.3638        | 12.0  | 17868 | 0.1088 | 4408      | 117314 | 5173       | 1.2336          | 0.2350 | 26459         | 0.2432 | 0.3764 | 0.6236 |
| 1.335         | 13.0  | 19357 | 0.1060 | 4868      | 117783 | 4467       | 1.2177          | 0.2284 | 25530         | 0.2353 | 0.3665 | 0.6335 |
| 1.2966        | 14.0  | 20846 | 0.1006 | 4561      | 119385 | 4599       | 1.1945          | 0.2186 | 24235         | 0.2254 | 0.3511 | 0.6489 |
| 1.2826        | 15.0  | 22335 | 0.0969 | 4455      | 119983 | 4111       | 1.1767          | 0.2122 | 23743         | 0.2180 | 0.3429 | 0.6571 |
| 1.2529        | 16.0  | 23824 | 0.0954 | 4229      | 120627 | 4312       | 1.1664          | 0.2090 | 23325         | 0.2150 | 0.3377 | 0.6623 |
| 1.2103        | 17.0  | 25313 | 0.0907 | 4218      | 121711 | 3998       | 1.1458          | 0.2002 | 22252         | 0.2056 | 0.3244 | 0.6756 |
| 1.18          | 18.0  | 26802 | 0.0891 | 4153      | 122139 | 3883       | 1.1355          | 0.1968 | 21889         | 0.2019 | 0.3194 | 0.6806 |
| 1.1529        | 19.0  | 28291 | 0.0864 | 4132      | 122860 | 3681       | 1.1236          | 0.1910 | 21189         | 0.1957 | 0.3105 | 0.6895 |
| 1.1342        | 20.0  | 29780 | 0.0855 | 4133      | 123172 | 3671       | 1.1174          | 0.1889 | 20876         | 0.1935 | 0.3069 | 0.6931 |


### Framework versions

- Transformers 4.40.0.dev0
- Pytorch 2.2.0+rocm5.6
- Datasets 2.18.0
- Tokenizers 0.15.2

### Wandb run
https://wandb.ai/butspeechfit/decred_commonvoice_en/runs/DeCRED_small_cv_restart