File size: 2,981 Bytes
5fc43a2 9edbe83 5fc43a2 9edbe83 5fc43a2 9edbe83 5fc43a2 9edbe83 5fc43a2 5c310a0 5fc43a2 a3404b9 5fc43a2 a3404b9 5fc43a2 a3404b9 512d511 3a631ed a3404b9 5fc43a2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 |
---
language:
- en
datasets:
- mozilla-foundation/common_voice_13_0
- facebook/voxpopuli
- LIUM/tedlium
- librispeech_asr
- fisher_corpus
- Switchboard-1
- WSJ-0
metrics:
- wer
pipeline_tag: automatic-speech-recognition
model-index:
- name: tbd
results:
- task:
type: automatic-speech-recognition
name: Automatic Speech Recognition
dataset:
name: LibriSpeech (clean)
type: librispeech_asr
config: other
split: test
args:
language: en
metrics:
- type: wer
value: 2.5
name: Test WER
- type: wer
value: 5.6
name: Test WER
- task:
type: Automatic Speech Recognition
name: automatic-speech-recognition
dataset:
name: tedlium-v3
type: LIUM/tedlium
config: release1
split: test
args:
language: en
metrics:
- type: wer
value: 6.3
name: Test WER
- task:
type: automatic-speech-recognition
name: Automatic Speech Recognition
dataset:
name: Vox Populi
type: facebook/voxpopuli
config: en
split: test
args:
language: en
metrics:
- type: wer
value: 7.3
name: Test WER
- task:
type: Automatic Speech Recognition
name: automatic-speech-recognition
dataset:
name: Mozilla Common Voice 13.0
type: mozilla-foundation/common_voice_13_0
config: en
split: test
args:
language: en
metrics:
- type: wer
value: 12.1
name: Test WER
---
# EBranchRegulaFormer
This is a **174M encoder-decoder Ebranchformer model** trained with an intermediate regularization technique on 6,000 hours of open-source English data.
It achieves Word Error Rates (WERs) comparable to `openai/whisper-medium.en` across multiple datasets with just 1/4 of the parameters.
Architecture details, training hyperparameters, and a description of the proposed technique will be added soon.
*Disclaimer: The model currently hallucinates on segments containing silence only, as it was previously not trained on such data. The fix will be added soon.*
The model can be used with the [`pipeline`](https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.AutomaticSpeechRecognitionPipeline)
class to transcribe audio files of arbitrary length.
```python
from transformers import pipeline
model_id = "BUT-FIT/EBranchRegulaFormer-medium"
pipe = pipeline("automatic-speech-recognition", model=model_id, feature_extractor=model_id, trust_remote_code=True)
# In newer versions of transformers (>4.31.0), there is a bug in the pipeline inference type.
# The warning can be ignored.
pipe.type = "seq2seq"
# Standard greedy decoding
result = pipe("audio.wav")
# Beam search decoding with joint CTC-autoregressive scorer
generation_config = pipe.model.generation_config
generation_config.ctc_weight = 0.3
generation_config.num_beams = 5
generation_config.ctc_margin = 0
result = pipe("audio.wav")
``` |