mHuBERT-147upper-sorbian

This model is a fine-tuned version of utter-project/mHuBERT-147 on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

• Loss: 3.2170
• Wer: 1.0
• Cer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
7.2839	3.9216	100	7.4925	1.0	1.0
3.4632	7.8431	200	3.4671	1.0	1.0
3.181	11.7647	300	3.2306	1.0	1.0
3.2284	15.6863	400	3.2231	1.0	1.0
3.2113	19.6078	500	3.2243	1.0	1.0
3.1844	23.5294	600	3.2183	1.0	1.0
3.2644	27.4510	700	3.2180	1.0	1.0
3.2111	31.3725	800	3.2191	1.0	1.0
3.187	35.2941	900	3.2189	1.0	1.0
3.2133	39.2157	1000	3.2203	1.0	1.0
3.2406	43.1373	1100	3.2181	1.0	1.0
3.1993	47.0588	1200	3.2178	1.0	1.0
3.2036	50.9804	1300	3.2169	1.0	1.0
3.2283	54.9020	1400	3.2171	1.0	1.0
3.1854	58.8235	1500	3.2198	1.0	1.0
3.184	62.7451	1600	3.2182	1.0	1.0
3.2253	66.6667	1700	3.2194	1.0	1.0
3.1943	70.5882	1800	3.2194	1.0	1.0
3.201	74.5098	1900	3.2167	1.0	1.0
3.2178	78.4314	2000	3.2180	1.0	1.0
3.2252	82.3529	2100	3.2172	1.0	1.0
3.2081	86.2745	2200	3.2170	1.0	1.0
3.2125	90.1961	2300	3.2170	1.0	1.0
3.23	94.1176	2400	3.2170	1.0	1.0
3.1851	98.0392	2500	3.2170	1.0	1.0

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1