DeCRED_small_cv_linear_mixing

This model is a fine-tuned version of Lakoc/DeCRED_small_cv on the common_voice_13_0 dataset. It achieves the following results on the evaluation set:

• Loss: 1.5358
• Cer: 0.1334
• Wer: 0.2616
• Mer: 0.2546
• Wil: 0.4037
• Wip: 0.5963
• Hits: 34174
• Substitutions: 8492
• Deletions: 1958
• Insertions: 1223

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 4
• total_train_batch_size: 512
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50.0

Training results

Training Loss	Epoch	Step	Validation Loss	Cer	Wer	Mer	Wil	Wip	Hits	Substitutions	Deletions	Insertions
3.0556	0.98	22	3.0178	1.2906	1.4035	0.9218	0.9902	0.0098	5310	35886	3428	23314
2.9712	2.0	45	2.9272	1.0482	1.2086	0.8876	0.9816	0.0184	6832	33809	3983	16140
2.8778	2.98	67	2.8449	0.8931	1.0776	0.8519	0.9699	0.0301	8362	31916	4346	11823
2.8329	4.0	90	2.7632	0.7772	0.9734	0.8097	0.9523	0.0477	10207	29712	4705	9021
2.7126	4.98	112	2.6890	0.6915	0.9029	0.7722	0.9331	0.0669	11886	27857	4881	7553
2.7523	6.0	135	2.6153	0.5987	0.8231	0.7289	0.9078	0.0922	13658	25936	5030	5764
2.5892	6.98	157	2.5484	0.5370	0.7637	0.6887	0.8801	0.1199	15403	24075	5146	4857
2.5555	8.0	180	2.4819	0.4797	0.7039	0.6449	0.8463	0.1537	17294	22237	5093	4083
2.4556	8.98	202	2.4215	0.4348	0.6531	0.6033	0.8103	0.1897	19160	20536	4928	3678
2.4722	10.0	225	2.3615	0.3908	0.5995	0.5587	0.7687	0.2313	21129	18865	4630	3255
2.3578	10.98	247	2.3072	0.3534	0.5561	0.5215	0.7310	0.2690	22766	17449	4409	2958
2.317	12.0	270	2.2533	0.3181	0.5166	0.4877	0.6952	0.3048	24219	16266	4139	2647
2.2757	12.98	292	2.2044	0.2935	0.4833	0.4587	0.6633	0.3367	25452	15275	3897	2393
2.2332	14.0	315	2.1560	0.2705	0.4522	0.4312	0.6319	0.3681	26614	14339	3671	2169
2.1887	14.98	337	2.1122	0.2526	0.4289	0.4106	0.6072	0.3928	27479	13605	3540	1996
2.1393	16.0	360	2.0689	0.2352	0.4049	0.3888	0.5811	0.4189	28408	12909	3307	1854
2.0646	16.98	382	2.0297	0.2213	0.3855	0.3712	0.5598	0.4402	29141	12367	3116	1719
2.0398	18.0	405	1.9911	0.2090	0.3687	0.3557	0.5407	0.4593	29799	11895	2930	1629
2.0011	18.98	427	1.9562	0.1994	0.3552	0.3432	0.5243	0.4757	30335	11455	2834	1561
2.0078	20.0	450	1.9219	0.1911	0.3435	0.3325	0.5108	0.4892	30776	11131	2717	1482
1.9608	20.98	472	1.8910	0.1843	0.3340	0.3236	0.4992	0.5008	31153	10840	2631	1435
1.9634	22.0	495	1.8605	0.1782	0.3258	0.3158	0.4888	0.5112	31498	10582	2544	1413
1.8888	22.98	517	1.8332	0.1730	0.3181	0.3086	0.4790	0.5210	31806	10335	2483	1375
1.9019	24.0	540	1.8064	0.1688	0.3118	0.3026	0.4711	0.5289	32063	10141	2420	1353
1.8107	24.98	562	1.7823	0.1660	0.3077	0.2987	0.4658	0.5342	32237	10016	2371	1342
1.8168	26.0	585	1.7588	0.1639	0.3029	0.2942	0.4598	0.5402	32427	9875	2322	1318
1.7804	26.98	607	1.7377	0.1612	0.2988	0.2903	0.4543	0.5457	32593	9734	2297	1301
1.8311	28.0	630	1.7171	0.1585	0.2954	0.2871	0.4499	0.5501	32726	9624	2274	1283
1.7236	28.98	652	1.6988	0.1564	0.2928	0.2846	0.4466	0.5534	32837	9549	2238	1278
1.741	30.0	675	1.6809	0.1548	0.2902	0.2822	0.4433	0.5567	32947	9474	2203	1274
1.7035	30.98	697	1.6650	0.1523	0.2880	0.2801	0.4403	0.5597	33036	9398	2190	1263
1.7518	32.0	720	1.6496	0.1504	0.2856	0.2778	0.4371	0.5629	33135	9320	2169	1255
1.6684	32.98	742	1.6360	0.1491	0.2837	0.2760	0.4345	0.5655	33214	9255	2155	1250
1.7251	34.0	765	1.6228	0.1474	0.2810	0.2734	0.4311	0.5689	33320	9175	2129	1236
1.6414	34.98	787	1.6112	0.1457	0.2788	0.2713	0.4280	0.5720	33416	9099	2109	1232
1.6707	36.0	810	1.6001	0.1444	0.2770	0.2696	0.4255	0.5745	33490	9034	2100	1229
1.6509	36.98	832	1.5904	0.1428	0.2749	0.2675	0.4225	0.5775	33588	8959	2077	1233
1.654	38.0	855	1.5812	0.1416	0.2732	0.2658	0.4199	0.5801	33670	8889	2065	1237
1.6305	38.98	877	1.5732	0.1403	0.2718	0.2645	0.4180	0.5820	33730	8841	2053	1235
1.6381	40.0	900	1.5658	0.1391	0.2703	0.2630	0.4158	0.5842	33811	8790	2023	1250
1.6393	40.98	922	1.5595	0.1382	0.2689	0.2616	0.4138	0.5862	33870	8741	2013	1246
1.6111	42.0	945	1.5537	0.1372	0.2675	0.2602	0.4117	0.5883	33937	8685	2002	1250
1.6041	42.98	967	1.5490	0.1362	0.2659	0.2587	0.4096	0.5904	34000	8636	1988	1240
1.6301	44.0	990	1.5448	0.1353	0.2647	0.2575	0.4079	0.5921	34052	8593	1979	1239
1.5736	44.98	1012	1.5416	0.1347	0.2638	0.2567	0.4066	0.5934	34087	8561	1976	1235
1.6209	46.0	1035	1.5389	0.1341	0.2626	0.2556	0.4050	0.5950	34135	8522	1967	1230
1.5892	46.98	1057	1.5372	0.1337	0.2620	0.2550	0.4042	0.5958	34156	8502	1966	1225
1.6066	48.0	1080	1.5361	0.1335	0.2617	0.2548	0.4039	0.5961	34167	8496	1961	1223
1.6445	48.89	1100	1.5358	0.1334	0.2616	0.2546	0.4037	0.5963	34174	8492	1958	1223

Framework versions

• Transformers 4.40.0.dev0
• Pytorch 2.2.0+rocm5.6
• Datasets 2.18.0
• Tokenizers 0.15.2

Wandb run

https://wandb.ai/butspeechfit/decred_commonvoice_en/runs/DeCRED_small_cv_linear_mixing