trip-fontaine
commited on
Commit
•
5265a3b
1
Parent(s):
4a16b87
readme update
Browse files
README.md
CHANGED
@@ -610,11 +610,11 @@ The distilled model performs to within 1% WER of Whisper large-v3 on out-of-dist
|
|
610 |
|
611 |
### Evaluation methodology
|
612 |
|
613 |
-
The model has been tested for both in-distribution (Common Voice 17 and Multilingual Librispeech) and out-of-distribution (Fleurs, Voxpopuli, custom [long-form test set](https://huggingface.co/datasets/speech-recognition-community-v2/dev_data)) short-form and long-form transcription performances.
|
614 |
|
615 |
**Short-form evaluations** are conducted on the four given datasets by first applying a filter to exclude samples longer than 30 seconds.
|
616 |
|
617 |
-
**Long-form evaluation** is conducted on a custom out-of-distribution [long-form test set](https://huggingface.co/datasets/eustlb/french-long-form-test).
|
618 |
|
619 |
### Short-Form
|
620 |
|
|
|
610 |
|
611 |
### Evaluation methodology
|
612 |
|
613 |
+
The model has been tested for both in-distribution (Common Voice 17 and Multilingual Librispeech) and out-of-distribution (Fleurs, Voxpopuli, custom [long-form test set](https://huggingface.co/datasets/speech-recognition-community-v2/dev_data)) short-form and long-form transcription performances. Models have been evaluated with SDPA, float32 and batch size 32.
|
614 |
|
615 |
**Short-form evaluations** are conducted on the four given datasets by first applying a filter to exclude samples longer than 30 seconds.
|
616 |
|
617 |
+
**Long-form evaluation** is conducted on a custom out-of-distribution [long-form test set](https://huggingface.co/datasets/eustlb/french-long-form-test) using OpenAI's sequential long-form transcription algorithm (see [Sequential Long-Form](#sequential-long-form) section) with long form generation parameters that can be found [here](https://github.com/huggingface/distil-whisper/blob/a5ed489ba6edb405ecef334ba0feec1bdca7a948/training/run_eval.py#L670C5-L676C6).
|
618 |
|
619 |
### Short-Form
|
620 |
|