Hugging Face's logo

BUT-FIT
/
whisper-ATC-czech-full

Automatic Speech Recognition
Transformers
PyTorch
TensorFlow
JAX
Safetensors
whisper
Inference Endpoints
Model card Files Community
whisper-ATC-czech-full
File size: 2,906 Bytes 
289ef62
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
0dc6445
289ef62
 
 
 
 
 
03d6080
289ef62
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6245d82
289ef62
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
---
license: apache-2.0
---


# Whisper Medium ATC full

This model is a fine-tuned [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on Czech and English air traffic communication recordings from Czech airport LKKU.

It was created as a product of bachelor's thesis at Faculty of Information Technology Brno University of Technology.

# Model description


- **Developed by:** Veronika Nevarilova ([@xnevar00](https://huggingface.co/xnevar00)), Igor Szoke ([@iszoke](https://huggingface.co/iszoke))
- **Shared by:** [BUT FIT](https://huggingface.co/BUT-FIT)
- **Model type:** Whisper
- **Languages:** Czech, English
- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
- **Finetuned from model:** [openai/whisper-medium](https://huggingface.co/openai/whisper-medium)


# Usage

```python
import torch
from transformers import pipeline

audio = "path/to/audio.xx"
device = "cuda:0" if torch.cuda.is_available() else "cpu"

transcribe = pipeline(task="automatic-speech-recognition", model="BUT-FIT/whisper-ATC-czech-full", chunk_length_s=30, device=device)
transcribe.model.config.forced_decoder_ids = transcribe.tokenizer.get_decoder_prompt_ids(task="transcribe", language="czech")
print('Transcription:', transcribe(audio)["text"])
```

# Dataset

Training dataset was made of ~5 hours of air traffic communication recordings. Recordings were Czech and English (80:20) and sporadically Slovak.

# Output format

The model was learned to transcribe every recording word by word. Transcription format of a recording is as follows:

Recording: *Oscar Kilo Alpha Bravo Charlie dráha dva nula střední pro přistání volná vítr nula jedna nula stupňů pět uzlů*

Transcription: `Oscar Kilo Alpha Bravo Charlie dráha dva nula střední pro přistání volná vítr nula jedna nula stupňů pět uzlů`

**Note:** See also model [BUT-FIT/whisper-ATC-czech-short](https://huggingface.co/BUT-FIT/whisper-ATC-czech-short), which abbreviates callsigns and numbers.


# Results

The model reached total WER of 14.7 % on unseen Czech and English LKKU recordings. 19.6 % WER was achieved on a testset containing Czech air traffic recordings from other airports, LKPR and LKTB.

WER of callsings in LKKU recordings was evaluated to be 6.2 %, while on LKPR and LKTB dataset the model reached 3.6 %.

# Training hyperparameters

- **learning_rate:** 3e-5
- **per_device_train_batch_size:** 2
- **gradient_accumulation_steps:** 8
- **warmup_ratio:** 0.12
- **fp16:** True
- **gradient_checkpointing:** True
- **evaluation_strategy:** "epoch"
- **save_strategy:** "epoch"
- **load_best_model_at_end:** True
- **metric_for_best_model:** "wer"
- **num_train_epochs:** 45

# Contact

For further information don't hesitate to contact Veronika Nevarilova (**[xnevar00@stud.fit.vutbr.cz](xnevar00@stud.fit.vutbr.cz)**) or Igor Szoke (**[szoke@fit.vutbr.cz](szoke@fit.vutbr.cz)**).