|
--- |
|
language: |
|
- sr |
|
base_model: openai/whisper-tiny |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- espnet/yodas |
|
- google/fleurs |
|
- classla/ParlaSpeech-RS |
|
- Sagicc/audio-lmb-ds |
|
- mozilla-foundation/common_voice_16_1 |
|
metrics: |
|
- wer |
|
model-index: |
|
- name: Whisper Tiny Sr Yodas |
|
results: |
|
- task: |
|
name: Automatic Speech Recognition |
|
type: automatic-speech-recognition |
|
dataset: |
|
name: Common Voice |
|
type: mozilla-foundation/common_voice_16_1 |
|
config: sr |
|
split: test |
|
args: sr |
|
metrics: |
|
- name: Wer |
|
type: wer |
|
value: 0.2187169545294325 |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# Whisper Tiny Sr Yodas |
|
|
|
This model is a fine-tuned version of [openai/whisper-timy](https://huggingface.co/openai/whisper-timy) on the Yodas dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.2420 |
|
- Wer Ortho: 0.3105 |
|
- Wer: 0.2187 |
|
|
|
## Model description |
|
|
|
This is just a test to run tiny version of Whisper. |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 1e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 50 |
|
- num_epochs: 10 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Wer Ortho | Wer | |
|
|:-------------:|:------:|:------:|:---------------:|:---------:|:------:| |
|
| 0.6063 | 0.0515 | 1000 | 0.4947 | 0.6804 | 0.5818 | |
|
| 0.5107 | 0.1031 | 2000 | 0.4324 | 0.5458 | 0.4501 | |
|
| 0.4468 | 0.1546 | 3000 | 0.3991 | 0.4992 | 0.4043 | |
|
| 0.4235 | 0.2061 | 4000 | 0.3721 | 0.5264 | 0.4293 | |
|
| 0.3878 | 0.2577 | 5000 | 0.3635 | 0.4590 | 0.3597 | |
|
| 0.3648 | 0.3092 | 6000 | 0.3377 | 0.4583 | 0.3599 | |
|
| 0.3434 | 0.3607 | 7000 | 0.3509 | 0.4904 | 0.4395 | |
|
| 0.381 | 0.4122 | 8000 | 0.3322 | 0.4249 | 0.3520 | |
|
| 0.3292 | 0.4638 | 9000 | 0.3321 | 0.5269 | 0.4787 | |
|
| 0.3529 | 0.5153 | 10000 | 0.3225 | 0.5528 | 0.4626 | |
|
| 0.3108 | 0.5668 | 11000 | 0.3107 | 0.4537 | 0.3576 | |
|
| 0.3474 | 0.6184 | 12000 | 0.3081 | 0.4616 | 0.3846 | |
|
| 0.2953 | 0.6699 | 13000 | 0.2941 | 0.4313 | 0.3558 | |
|
| 0.3326 | 0.7214 | 14000 | 0.2929 | 0.4364 | 0.3613 | |
|
| 0.2758 | 0.7730 | 15000 | 0.2942 | 0.4177 | 0.3525 | |
|
| 0.2817 | 0.8245 | 16000 | 0.2823 | 0.4576 | 0.3666 | |
|
| 0.2809 | 0.8760 | 17000 | 0.2867 | 0.4124 | 0.3119 | |
|
| 0.2924 | 0.9275 | 18000 | 0.2780 | 0.3796 | 0.2827 | |
|
| 0.2955 | 0.9791 | 19000 | 0.2745 | 0.3883 | 0.2908 | |
|
| 0.261 | 1.0306 | 20000 | 0.2728 | 0.3766 | 0.3026 | |
|
| 0.2424 | 1.0821 | 21000 | 0.2762 | 0.3793 | 0.3082 | |
|
| 0.2526 | 1.1337 | 22000 | 0.2701 | 0.3542 | 0.2635 | |
|
| 0.2428 | 1.1852 | 23000 | 0.2679 | 0.3766 | 0.2827 | |
|
| 0.2536 | 1.2367 | 24000 | 0.2690 | 0.3868 | 0.2860 | |
|
| 0.2433 | 1.2883 | 25000 | 0.2709 | 0.3838 | 0.2913 | |
|
| 0.2768 | 1.3398 | 26000 | 0.2623 | 0.3786 | 0.2816 | |
|
| 0.2684 | 1.3913 | 27000 | 0.2643 | 0.3858 | 0.2929 | |
|
| 0.2334 | 1.4429 | 28000 | 0.2580 | 0.3720 | 0.2774 | |
|
| 0.2441 | 1.4944 | 29000 | 0.2537 | 0.3526 | 0.2538 | |
|
| 0.2367 | 1.5459 | 30000 | 0.2519 | 0.3470 | 0.2541 | |
|
| 0.2491 | 1.5974 | 31000 | 0.2579 | 0.3528 | 0.2563 | |
|
| 0.2363 | 1.6490 | 32000 | 0.2584 | 0.3641 | 0.2681 | |
|
| 0.2475 | 1.7005 | 33000 | 0.2570 | 0.3743 | 0.2813 | |
|
| 0.2345 | 1.7520 | 34000 | 0.2540 | 0.3826 | 0.2920 | |
|
| 0.2359 | 1.8036 | 35000 | 0.2543 | 0.3879 | 0.3137 | |
|
| 0.2303 | 1.8551 | 36000 | 0.2456 | 0.3410 | 0.2670 | |
|
| 0.2403 | 1.9066 | 37000 | 0.2455 | 0.3722 | 0.2755 | |
|
| 0.2272 | 1.9582 | 38000 | 0.2499 | 0.3343 | 0.2420 | |
|
| 0.2114 | 2.0097 | 39000 | 0.2433 | 0.3196 | 0.2446 | |
|
| 0.1956 | 2.0612 | 40000 | 0.2500 | 0.3442 | 0.2741 | |
|
| 0.1968 | 2.1127 | 41000 | 0.2427 | 0.3270 | 0.2360 | |
|
| 0.2121 | 2.1643 | 42000 | 0.2477 | 0.3561 | 0.2686 | |
|
| 0.2017 | 2.2158 | 43000 | 0.2419 | 0.3442 | 0.2508 | |
|
| 0.2212 | 2.2673 | 44000 | 0.2448 | 0.4140 | 0.3195 | |
|
| 0.2302 | 2.3189 | 45000 | 0.2447 | 0.3697 | 0.2954 | |
|
| 0.2266 | 2.3704 | 46000 | 0.2471 | 0.3830 | 0.2887 | |
|
| 0.2039 | 2.4219 | 47000 | 0.2403 | 0.3299 | 0.2372 | |
|
| 0.1962 | 2.4735 | 48000 | 0.2395 | 0.3244 | 0.2312 | |
|
| 0.1941 | 2.5250 | 49000 | 0.2422 | 0.3427 | 0.2511 | |
|
| 0.1858 | 2.5765 | 50000 | 0.2375 | 0.3258 | 0.2351 | |
|
| 0.2147 | 2.6281 | 51000 | 0.2431 | 0.3417 | 0.2531 | |
|
| 0.2116 | 2.6796 | 52000 | 0.2407 | 0.3741 | 0.2838 | |
|
| 0.2216 | 2.7311 | 53000 | 0.2346 | 0.3295 | 0.2356 | |
|
| 0.2126 | 2.7826 | 54000 | 0.2360 | 0.3336 | 0.2453 | |
|
| 0.1999 | 2.8342 | 55000 | 0.2335 | 0.3147 | 0.2388 | |
|
| 0.2026 | 2.8857 | 56000 | 0.2314 | 0.3524 | 0.2800 | |
|
| 0.2038 | 2.9372 | 57000 | 0.2358 | 0.3412 | 0.2490 | |
|
| 0.2065 | 2.9888 | 58000 | 0.2352 | 0.3276 | 0.2326 | |
|
| 0.1903 | 3.0403 | 59000 | 0.2337 | 0.3216 | 0.2266 | |
|
| 0.1967 | 3.0918 | 60000 | 0.2335 | 0.3188 | 0.2222 | |
|
| 0.1641 | 3.1434 | 61000 | 0.2344 | 0.3242 | 0.2353 | |
|
| 0.1931 | 3.1949 | 62000 | 0.2375 | 0.3711 | 0.2767 | |
|
| 0.1935 | 3.2464 | 63000 | 0.2334 | 0.3429 | 0.2519 | |
|
| 0.1975 | 3.2979 | 64000 | 0.2351 | 0.3138 | 0.2189 | |
|
| 0.1793 | 3.3495 | 65000 | 0.2349 | 0.3145 | 0.2434 | |
|
| 0.1913 | 3.4010 | 66000 | 0.2388 | 0.3634 | 0.2700 | |
|
| 0.1874 | 3.4525 | 67000 | 0.2419 | 0.4172 | 0.3241 | |
|
| 0.2069 | 3.5041 | 68000 | 0.2303 | 0.3516 | 0.2582 | |
|
| 0.1918 | 3.5556 | 69000 | 0.2371 | 0.3674 | 0.2753 | |
|
| 0.1776 | 3.6071 | 70000 | 0.2319 | 0.3265 | 0.2356 | |
|
| 0.1905 | 3.6587 | 71000 | 0.2270 | 0.3202 | 0.2286 | |
|
| 0.1813 | 3.7102 | 72000 | 0.2331 | 0.3195 | 0.2266 | |
|
| 0.1977 | 3.7617 | 73000 | 0.2296 | 0.3442 | 0.2504 | |
|
| 0.1915 | 3.8133 | 74000 | 0.2298 | 0.3442 | 0.2504 | |
|
| 0.1866 | 3.8648 | 75000 | 0.2312 | 0.3235 | 0.2328 | |
|
| 0.2026 | 3.9163 | 76000 | 0.2282 | 0.3046 | 0.2096 | |
|
| 0.1809 | 3.9678 | 77000 | 0.2310 | 0.3581 | 0.2670 | |
|
| 0.1878 | 4.0194 | 78000 | 0.2321 | 0.3101 | 0.2173 | |
|
| 0.1632 | 4.0709 | 79000 | 0.2396 | 0.3346 | 0.2425 | |
|
| 0.1812 | 4.1224 | 80000 | 0.2313 | 0.3685 | 0.2742 | |
|
| 0.1738 | 4.1740 | 81000 | 0.2296 | 0.3466 | 0.2557 | |
|
| 0.161 | 4.2255 | 82000 | 0.2299 | 0.3018 | 0.2096 | |
|
| 0.175 | 4.2770 | 83000 | 0.2344 | 0.3150 | 0.2214 | |
|
| 0.1822 | 4.3286 | 84000 | 0.2309 | 0.3061 | 0.2111 | |
|
| 0.1559 | 4.3801 | 85000 | 0.2304 | 0.3120 | 0.2200 | |
|
| 0.1679 | 4.4316 | 86000 | 0.2315 | 0.2981 | 0.2048 | |
|
| 0.1579 | 4.4831 | 87000 | 0.2311 | 0.3233 | 0.2309 | |
|
| 0.16 | 4.5347 | 88000 | 0.2307 | 0.3094 | 0.2159 | |
|
| 0.1601 | 4.5862 | 89000 | 0.2307 | 0.3142 | 0.2205 | |
|
| 0.1536 | 4.6377 | 90000 | 0.2343 | 0.3537 | 0.2610 | |
|
| 0.1757 | 4.6893 | 91000 | 0.2344 | 0.3424 | 0.2504 | |
|
| 0.1468 | 4.7408 | 92000 | 0.2306 | 0.3073 | 0.2159 | |
|
| 0.1672 | 4.7923 | 93000 | 0.2282 | 0.3486 | 0.2563 | |
|
| 0.1565 | 4.8439 | 94000 | 0.2272 | 0.3082 | 0.2180 | |
|
| 0.1732 | 4.8954 | 95000 | 0.2258 | 0.3177 | 0.2281 | |
|
| 0.1707 | 4.9469 | 96000 | 0.2315 | 0.3554 | 0.2621 | |
|
| 0.1655 | 4.9985 | 97000 | 0.2259 | 0.3106 | 0.2192 | |
|
| 0.1564 | 5.0500 | 98000 | 0.2295 | 0.3016 | 0.2078 | |
|
| 0.1465 | 5.1015 | 99000 | 0.2310 | 0.3020 | 0.2097 | |
|
| 0.1534 | 5.1530 | 100000 | 0.2334 | 0.3158 | 0.2210 | |
|
| 0.163 | 5.2046 | 101000 | 0.2324 | 0.3186 | 0.2265 | |
|
| 0.1656 | 5.2561 | 102000 | 0.2309 | 0.3068 | 0.2122 | |
|
| 0.1459 | 5.3076 | 103000 | 0.2333 | 0.3136 | 0.2221 | |
|
| 0.1539 | 5.3592 | 104000 | 0.2346 | 0.3149 | 0.2228 | |
|
| 0.1551 | 5.4107 | 105000 | 0.2296 | 0.3572 | 0.2659 | |
|
| 0.1378 | 5.4622 | 106000 | 0.2324 | 0.3249 | 0.2318 | |
|
| 0.1562 | 5.5138 | 107000 | 0.2321 | 0.3216 | 0.2261 | |
|
| 0.1622 | 5.5653 | 108000 | 0.2324 | 0.3143 | 0.2226 | |
|
| 0.1524 | 5.6168 | 109000 | 0.2282 | 0.3013 | 0.2088 | |
|
| 0.1501 | 5.6683 | 110000 | 0.2309 | 0.3080 | 0.2177 | |
|
| 0.1434 | 5.7199 | 111000 | 0.2294 | 0.3016 | 0.2104 | |
|
| 0.1594 | 5.7714 | 112000 | 0.2314 | 0.3011 | 0.2118 | |
|
| 0.144 | 5.8229 | 113000 | 0.2295 | 0.3069 | 0.2170 | |
|
| 0.1513 | 5.8745 | 114000 | 0.2286 | 0.3053 | 0.2129 | |
|
| 0.1575 | 5.9260 | 115000 | 0.2302 | 0.3085 | 0.2152 | |
|
| 0.153 | 5.9775 | 116000 | 0.2305 | 0.3009 | 0.2039 | |
|
| 0.1518 | 6.0291 | 117000 | 0.2314 | 0.2986 | 0.2066 | |
|
| 0.1367 | 6.0806 | 118000 | 0.2324 | 0.3182 | 0.2233 | |
|
| 0.1383 | 6.1321 | 119000 | 0.2322 | 0.3404 | 0.2460 | |
|
| 0.1286 | 6.1837 | 120000 | 0.2324 | 0.3009 | 0.2274 | |
|
| 0.1236 | 6.2352 | 121000 | 0.2320 | 0.3066 | 0.2154 | |
|
| 0.1395 | 6.2867 | 122000 | 0.2327 | 0.3071 | 0.2175 | |
|
| 0.1481 | 6.3382 | 123000 | 0.2325 | 0.3055 | 0.2155 | |
|
| 0.143 | 6.3898 | 124000 | 0.2328 | 0.3073 | 0.2125 | |
|
| 0.1399 | 6.4413 | 125000 | 0.2345 | 0.3112 | 0.2207 | |
|
| 0.1256 | 6.4928 | 126000 | 0.2312 | 0.3193 | 0.2261 | |
|
| 0.131 | 6.5444 | 127000 | 0.2315 | 0.3191 | 0.2266 | |
|
| 0.1529 | 6.5959 | 128000 | 0.2332 | 0.3115 | 0.2182 | |
|
| 0.1568 | 6.6474 | 129000 | 0.2331 | 0.3080 | 0.2180 | |
|
| 0.146 | 6.6990 | 130000 | 0.2328 | 0.3066 | 0.2122 | |
|
| 0.1536 | 6.7505 | 131000 | 0.2326 | 0.3022 | 0.2078 | |
|
| 0.1394 | 6.8020 | 132000 | 0.2316 | 0.3046 | 0.2125 | |
|
| 0.1406 | 6.8536 | 133000 | 0.2346 | 0.3182 | 0.2254 | |
|
| 0.1373 | 6.9051 | 134000 | 0.2311 | 0.3062 | 0.2148 | |
|
| 0.1479 | 6.9566 | 135000 | 0.2322 | 0.2971 | 0.2080 | |
|
| 0.1408 | 7.0081 | 136000 | 0.2359 | 0.3031 | 0.2124 | |
|
| 0.1369 | 7.0597 | 137000 | 0.2385 | 0.3053 | 0.2134 | |
|
| 0.1309 | 7.1112 | 138000 | 0.2350 | 0.3108 | 0.2196 | |
|
| 0.1402 | 7.1627 | 139000 | 0.2352 | 0.3071 | 0.2157 | |
|
| 0.1425 | 7.2143 | 140000 | 0.2378 | 0.3083 | 0.2154 | |
|
| 0.1428 | 7.2658 | 141000 | 0.2357 | 0.3098 | 0.2164 | |
|
| 0.1401 | 7.3173 | 142000 | 0.2386 | 0.3168 | 0.2207 | |
|
| 0.1326 | 7.3689 | 143000 | 0.2365 | 0.3078 | 0.2175 | |
|
| 0.1211 | 7.4204 | 144000 | 0.2360 | 0.3006 | 0.2108 | |
|
| 0.1369 | 7.4719 | 145000 | 0.2348 | 0.3083 | 0.2175 | |
|
| 0.1272 | 7.5234 | 146000 | 0.2347 | 0.3103 | 0.2203 | |
|
| 0.1253 | 7.5750 | 147000 | 0.2371 | 0.3075 | 0.2131 | |
|
| 0.1128 | 7.6265 | 148000 | 0.2371 | 0.3087 | 0.2175 | |
|
| 0.1429 | 7.6780 | 149000 | 0.2354 | 0.3053 | 0.2103 | |
|
| 0.1307 | 7.7296 | 150000 | 0.2352 | 0.3115 | 0.2180 | |
|
| 0.1272 | 7.7811 | 151000 | 0.2375 | 0.3027 | 0.2092 | |
|
| 0.1364 | 7.8326 | 152000 | 0.2364 | 0.3096 | 0.2185 | |
|
| 0.1285 | 7.8842 | 153000 | 0.2341 | 0.3046 | 0.2131 | |
|
| 0.1359 | 7.9357 | 154000 | 0.2361 | 0.3057 | 0.2154 | |
|
| 0.1345 | 7.9872 | 155000 | 0.2352 | 0.2937 | 0.2025 | |
|
| 0.1252 | 8.0388 | 156000 | 0.2383 | 0.3050 | 0.2131 | |
|
| 0.1248 | 8.0903 | 157000 | 0.2399 | 0.3161 | 0.2221 | |
|
| 0.1131 | 8.1418 | 158000 | 0.2383 | 0.3115 | 0.2185 | |
|
| 0.1209 | 8.1933 | 159000 | 0.2391 | 0.3115 | 0.2191 | |
|
| 0.1076 | 8.2449 | 160000 | 0.2384 | 0.3140 | 0.2198 | |
|
| 0.1378 | 8.2964 | 161000 | 0.2384 | 0.3099 | 0.2184 | |
|
| 0.1265 | 8.3479 | 162000 | 0.2413 | 0.3235 | 0.2291 | |
|
| 0.1256 | 8.3995 | 163000 | 0.2396 | 0.3129 | 0.2185 | |
|
| 0.1325 | 8.4510 | 164000 | 0.2404 | 0.3073 | 0.2138 | |
|
| 0.1232 | 8.5025 | 165000 | 0.2398 | 0.3119 | 0.2212 | |
|
| 0.1264 | 8.5541 | 166000 | 0.2398 | 0.3082 | 0.2159 | |
|
| 0.1146 | 8.6056 | 167000 | 0.2412 | 0.3083 | 0.2170 | |
|
| 0.1338 | 8.6571 | 168000 | 0.2384 | 0.3064 | 0.2143 | |
|
| 0.1302 | 8.7086 | 169000 | 0.2395 | 0.3082 | 0.2168 | |
|
| 0.1179 | 8.7602 | 170000 | 0.2402 | 0.3087 | 0.2175 | |
|
| 0.1267 | 8.8117 | 171000 | 0.2404 | 0.3096 | 0.2182 | |
|
| 0.1361 | 8.8632 | 172000 | 0.2413 | 0.3122 | 0.2219 | |
|
| 0.1204 | 8.9148 | 173000 | 0.2379 | 0.3071 | 0.2148 | |
|
| 0.1299 | 8.9663 | 174000 | 0.2388 | 0.3122 | 0.2205 | |
|
| 0.1158 | 9.0178 | 175000 | 0.2401 | 0.3131 | 0.2191 | |
|
| 0.1181 | 9.0694 | 176000 | 0.2403 | 0.3050 | 0.2129 | |
|
| 0.1219 | 9.1209 | 177000 | 0.2425 | 0.3082 | 0.2145 | |
|
| 0.1203 | 9.1724 | 178000 | 0.2413 | 0.3052 | 0.2122 | |
|
| 0.1141 | 9.2240 | 179000 | 0.2423 | 0.3128 | 0.2196 | |
|
| 0.114 | 9.2755 | 180000 | 0.2428 | 0.3122 | 0.2201 | |
|
| 0.1231 | 9.3270 | 181000 | 0.2418 | 0.3101 | 0.2185 | |
|
| 0.1108 | 9.3785 | 182000 | 0.2411 | 0.3094 | 0.2177 | |
|
| 0.1135 | 9.4301 | 183000 | 0.2419 | 0.3098 | 0.2173 | |
|
| 0.1225 | 9.4816 | 184000 | 0.2422 | 0.3119 | 0.2201 | |
|
| 0.1264 | 9.5331 | 185000 | 0.2417 | 0.3073 | 0.2136 | |
|
| 0.117 | 9.5847 | 186000 | 0.2429 | 0.3061 | 0.2131 | |
|
| 0.1273 | 9.6362 | 187000 | 0.2422 | 0.3076 | 0.2145 | |
|
| 0.1275 | 9.6877 | 188000 | 0.2420 | 0.3101 | 0.2180 | |
|
| 0.1178 | 9.7393 | 189000 | 0.2422 | 0.3112 | 0.2194 | |
|
| 0.1186 | 9.7908 | 190000 | 0.2421 | 0.3136 | 0.2212 | |
|
| 0.1134 | 9.8423 | 191000 | 0.2420 | 0.3105 | 0.2187 | |
|
| 0.1202 | 9.8938 | 192000 | 0.2423 | 0.3106 | 0.2189 | |
|
| 0.1225 | 9.9454 | 193000 | 0.2422 | 0.3106 | 0.2189 | |
|
| 0.1328 | 9.9969 | 194000 | 0.2420 | 0.3105 | 0.2187 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.40.1 |
|
- Pytorch 2.3.0+cu121 |
|
- Datasets 2.19.0 |
|
- Tokenizers 0.19.1 |
