---
language:
- ga
- en
license: apache-2.0
base_model: openai/whisper-small
tags:
- generated_from_trainer
datasets:
- ymoslem/IWSLT2023-GA-EN
- ymoslem/FLEURS-GA-EN
- ymoslem/BitesizeIrish-GA-EN
- ymoslem/SpokenWords-GA-EN-MTed
metrics:
- bleu
- wer
model-index:
- name: Whisper Small GA-EN Speech Translation + VAD
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: IWSLT-2023, FLEURS, BiteSize, and SpokenWords
      type: ymoslem/IWSLT2023-GA-EN
    metrics:
    - name: Bleu
      type: bleu
      value: 28.22
    - name: Wer
      type: wer
      value: 68.52769022962629
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Whisper Small GA-EN Speech Translation + VAD

This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on the IWSLT-2023, FLEURS, BiteSize, and SpokenWords dataset.
It achieves the following results on the evaluation set:
- Loss: 1.7352
- Bleu: 28.22
- Chrf: 44.19
- Wer: 68.5277

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 3000
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Bleu  | Chrf  | Wer      |
|:-------------:|:------:|:----:|:---------------:|:-----:|:-----:|:--------:|
| 1.9529        | 0.2188 | 100  | 1.7388          | 12.76 | 29.03 | 97.1184  |
| 1.5762        | 0.4376 | 200  | 1.5362          | 15.3  | 33.31 | 98.4241  |
| 1.2624        | 0.6565 | 300  | 1.4346          | 17.94 | 37.2  | 101.4408 |
| 1.0367        | 0.8753 | 400  | 1.4502          | 21.52 | 39.13 | 85.4120  |
| 0.4677        | 1.0941 | 500  | 1.4693          | 23.26 | 40.49 | 78.4331  |
| 0.4284        | 1.3129 | 600  | 1.5163          | 21.31 | 41.41 | 86.0873  |
| 0.4026        | 1.5317 | 700  | 1.4999          | 24.11 | 40.59 | 79.3787  |
| 0.4132        | 1.7505 | 800  | 1.5134          | 27.77 | 43.01 | 70.1936  |
| 0.3701        | 1.9694 | 900  | 1.5368          | 27.74 | 42.61 | 66.0964  |
| 0.1337        | 2.1882 | 1000 | 1.5692          | 27.96 | 43.77 | 64.9257  |
| 0.143         | 2.4070 | 1100 | 1.5516          | 26.06 | 42.12 | 71.3192  |
| 0.144         | 2.6258 | 1200 | 1.5839          | 27.55 | 43.19 | 69.7434  |
| 0.1372        | 2.8446 | 1300 | 1.5510          | 27.93 | 43.07 | 66.1414  |
| 0.0573        | 3.0635 | 1400 | 1.6567          | 26.34 | 41.69 | 72.3998  |
| 0.0554        | 3.2823 | 1500 | 1.6511          | 27.98 | 42.66 | 68.5277  |
| 0.0534        | 3.5011 | 1600 | 1.6732          | 28.29 | 43.2  | 67.1319  |
| 0.0588        | 3.7199 | 1700 | 1.6687          | 27.0  | 43.31 | 70.7789  |
| 0.0486        | 3.9387 | 1800 | 1.6759          | 28.02 | 43.97 | 66.3665  |
| 0.0224        | 4.1575 | 1900 | 1.7597          | 26.86 | 41.81 | 70.5538  |
| 0.0264        | 4.3764 | 2000 | 1.7113          | 27.58 | 43.38 | 70.4638  |
| 0.0233        | 4.5952 | 2100 | 1.7013          | 27.83 | 42.87 | 68.2575  |
| 0.0192        | 4.8140 | 2200 | 1.7351          | 25.39 | 42.09 | 78.0279  |
| 0.0149        | 5.0328 | 2300 | 1.7350          | 27.62 | 43.99 | 70.5538  |
| 0.0086        | 5.2516 | 2400 | 1.7331          | 29.37 | 45.08 | 68.5277  |
| 0.006         | 5.4705 | 2500 | 1.7145          | 29.04 | 44.19 | 66.9968  |
| 0.0064        | 5.6893 | 2600 | 1.7322          | 28.27 | 43.6  | 70.2386  |
| 0.0053        | 5.9081 | 2700 | 1.7239          | 27.86 | 43.78 | 69.6083  |
| 0.0021        | 6.1269 | 2800 | 1.7288          | 28.14 | 44.12 | 68.5727  |
| 0.0016        | 6.3457 | 2900 | 1.7375          | 28.26 | 44.14 | 68.7078  |
| 0.0023        | 6.5646 | 3000 | 1.7352          | 28.22 | 44.19 | 68.5277  |


### Framework versions

- Transformers 4.41.1
- Pytorch 2.2.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1