DGurgurov/maltese-wiki-lang-adapter

mt

This model is a fine-tuned version of bert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0345
• Accuracy: 0.7947

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 50000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.8769	0.39	500	2.3415	0.5941
2.3422	0.78	1000	2.0567	0.6324
2.1192	1.17	1500	1.8960	0.6535
1.9894	1.57	2000	1.7869	0.6695
1.8961	1.96	2500	1.7181	0.6796
1.8258	2.35	3000	1.6541	0.6893
1.7648	2.74	3500	1.5881	0.6996
1.7159	3.13	4000	1.5544	0.7065
1.6763	3.52	4500	1.5229	0.7101
1.6338	3.92	5000	1.4794	0.7166
1.6	4.31	5500	1.4452	0.7222
1.5832	4.7	6000	1.4302	0.7250
1.5532	5.09	6500	1.4013	0.7308
1.5247	5.48	7000	1.3956	0.7325
1.5103	5.87	7500	1.3598	0.7367
1.4866	6.26	8000	1.3331	0.7401
1.468	6.66	8500	1.3266	0.7428
1.4402	7.05	9000	1.3119	0.7457
1.4255	7.44	9500	1.2967	0.7481
1.4236	7.83	10000	1.2779	0.7516
1.41	8.22	10500	1.2598	0.7544
1.3994	8.61	11000	1.2677	0.7539
1.3809	9.01	11500	1.2334	0.7579
1.3689	9.4	12000	1.2468	0.7581
1.3637	9.79	12500	1.2349	0.7588
1.3587	10.18	13000	1.2157	0.7625
1.3397	10.57	13500	1.2055	0.7630
1.3347	10.96	14000	1.1968	0.7654
1.315	11.35	14500	1.1955	0.7652
1.3246	11.75	15000	1.1886	0.7674
1.3078	12.14	15500	1.1942	0.7660
1.2925	12.53	16000	1.1850	0.7678
1.3004	12.92	16500	1.1747	0.7692
1.2911	13.31	17000	1.1591	0.7719
1.2786	13.7	17500	1.1602	0.7734
1.2771	14.1	18000	1.1597	0.7717
1.2774	14.49	18500	1.1547	0.7724
1.2652	14.88	19000	1.1403	0.7751
1.262	15.27	19500	1.1397	0.7754
1.2595	15.66	20000	1.1325	0.7778
1.2544	16.05	20500	1.1385	0.7759
1.2424	16.44	21000	1.1291	0.7774
1.2361	16.84	21500	1.1338	0.7782
1.2325	17.23	22000	1.1081	0.7818
1.236	17.62	22500	1.1161	0.7789
1.2284	18.01	23000	1.1150	0.7809
1.2267	18.4	23500	1.1001	0.7831
1.2151	18.79	24000	1.1054	0.7829
1.2197	19.19	24500	1.1096	0.7814
1.2226	19.58	25000	1.1098	0.7815
1.2101	19.97	25500	1.0962	0.7840
1.2102	20.36	26000	1.0920	0.7847
1.2003	20.75	26500	1.0828	0.7863
1.1912	21.14	27000	1.0886	0.7854
1.1987	21.53	27500	1.0860	0.7860
1.2072	21.93	28000	1.0812	0.7859
1.1894	22.32	28500	1.0816	0.7858
1.2031	22.71	29000	1.0771	0.7874
1.1819	23.1	29500	1.0674	0.7881
1.185	23.49	30000	1.0761	0.7879
1.1873	23.88	30500	1.0697	0.7892
1.1793	24.28	31000	1.0706	0.7884
1.1793	24.67	31500	1.0622	0.7899
1.1748	25.06	32000	1.0630	0.7894
1.1701	25.45	32500	1.0643	0.7889
1.1678	25.84	33000	1.0567	0.7906
1.177	26.23	33500	1.0660	0.7886
1.1749	26.62	34000	1.0652	0.7911
1.1623	27.02	34500	1.0436	0.7924
1.1647	27.41	35000	1.0769	0.7873
1.1692	27.8	35500	1.0474	0.7918
1.1572	28.19	36000	1.0454	0.7922
1.1612	28.58	36500	1.0554	0.7916
1.1626	28.97	37000	1.0492	0.7918
1.1613	29.37	37500	1.0586	0.7909
1.146	29.76	38000	1.0470	0.7918
1.1558	30.15	38500	1.0530	0.7921
1.1553	30.54	39000	1.0474	0.7910
1.1543	30.93	39500	1.0446	0.7920
1.1523	31.32	40000	1.0521	0.7916
1.1529	31.71	40500	1.0489	0.7923
1.1528	32.11	41000	1.0407	0.7930
1.1532	32.5	41500	1.0386	0.7943
1.1415	32.89	42000	1.0489	0.7913
1.1509	33.28	42500	1.0355	0.7940
1.1484	33.67	43000	1.0375	0.7931
1.1434	34.06	43500	1.0431	0.7928
1.1464	34.46	44000	1.0348	0.7949
1.1394	34.85	44500	1.0514	0.7927
1.1418	35.24	45000	1.0429	0.7933
1.1453	35.63	45500	1.0423	0.7942
1.1411	36.02	46000	1.0358	0.7949
1.1434	36.41	46500	1.0308	0.7954
1.1392	36.81	47000	1.0326	0.7950
1.137	37.2	47500	1.0315	0.7948
1.14	37.59	48000	1.0406	0.7937
1.142	37.98	48500	1.0464	0.7933
1.1404	38.37	49000	1.0423	0.7933
1.1412	38.76	49500	1.0363	0.7950
1.143	39.15	50000	1.0355	0.7950

Framework versions

• Transformers 4.36.2
• Pytorch 2.0.0
• Datasets 2.15.0
• Tokenizers 0.15.0