bert-multi-base-uncased-finetuned-pos-ky

This model is a fine-tuned version of google-bert/bert-base-multilingual-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0007
• Precision: 0.8230
• Recall: 0.8280
• F1: 0.8255
• Accuracy: 0.8850

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	40	0.6248	0.6903	0.6708	0.6804	0.8350
No log	2.0	80	0.4938	0.7555	0.7555	0.7555	0.8626
No log	3.0	120	0.4931	0.7939	0.7948	0.7944	0.8764
No log	4.0	160	0.4948	0.7776	0.8034	0.7903	0.8735
No log	5.0	200	0.4744	0.8102	0.8231	0.8166	0.8850
No log	6.0	240	0.5698	0.8042	0.8071	0.8056	0.8787
No log	7.0	280	0.5787	0.7878	0.8120	0.7998	0.8758
No log	8.0	320	0.6357	0.7841	0.8120	0.7978	0.8718
No log	9.0	360	0.6359	0.8265	0.8366	0.8315	0.8879
No log	10.0	400	0.6735	0.8048	0.8305	0.8174	0.8827
No log	11.0	440	0.7243	0.8087	0.8206	0.8146	0.8804
No log	12.0	480	0.7430	0.8133	0.8292	0.8212	0.8827
0.244	13.0	520	0.7097	0.8058	0.8206	0.8131	0.8810
0.244	14.0	560	0.7885	0.8152	0.8182	0.8167	0.8787
0.244	15.0	600	0.7925	0.8082	0.8231	0.8156	0.8827
0.244	16.0	640	0.7850	0.8270	0.8280	0.8275	0.8879
0.244	17.0	680	0.7881	0.8162	0.8292	0.8227	0.8850
0.244	18.0	720	0.8490	0.8168	0.8219	0.8194	0.8810
0.244	19.0	760	0.8470	0.8163	0.8243	0.8203	0.8815
0.244	20.0	800	0.8792	0.8007	0.8194	0.8100	0.8752
0.244	21.0	840	0.9056	0.8084	0.8243	0.8163	0.8769
0.244	22.0	880	0.9099	0.8152	0.8292	0.8222	0.8827
0.244	23.0	920	0.8455	0.8166	0.8317	0.8241	0.8844
0.244	24.0	960	0.9336	0.8140	0.8170	0.8155	0.8775
0.0193	25.0	1000	0.9462	0.8145	0.8145	0.8145	0.8787
0.0193	26.0	1040	0.9457	0.8200	0.8170	0.8185	0.8792
0.0193	27.0	1080	0.9312	0.8177	0.8268	0.8222	0.8798
0.0193	28.0	1120	0.9553	0.8235	0.8194	0.8214	0.8833
0.0193	29.0	1160	0.9450	0.8207	0.8268	0.8237	0.8821
0.0193	30.0	1200	0.9337	0.8335	0.8366	0.8351	0.8896
0.0193	31.0	1240	0.9476	0.8203	0.8354	0.8278	0.8861
0.0193	32.0	1280	0.9443	0.8182	0.8292	0.8237	0.8838
0.0193	33.0	1320	0.9713	0.8197	0.8268	0.8232	0.8844
0.0193	34.0	1360	0.9751	0.8210	0.8280	0.8245	0.8821
0.0193	35.0	1400	0.9850	0.8129	0.8219	0.8173	0.8815
0.0193	36.0	1440	0.9546	0.8182	0.8292	0.8237	0.8821
0.0193	37.0	1480	0.9713	0.8216	0.8317	0.8266	0.8844
0.0049	38.0	1520	0.9696	0.8234	0.8305	0.8269	0.8850
0.0049	39.0	1560	0.9722	0.8222	0.8354	0.8288	0.8861
0.0049	40.0	1600	0.9705	0.8273	0.8354	0.8313	0.8879
0.0049	41.0	1640	0.9777	0.8190	0.8280	0.8235	0.8838
0.0049	42.0	1680	0.9841	0.8167	0.8268	0.8217	0.8850
0.0049	43.0	1720	0.9799	0.8234	0.8305	0.8269	0.8873
0.0049	44.0	1760	0.9785	0.8248	0.8329	0.8289	0.8873
0.0049	45.0	1800	0.9863	0.8205	0.8256	0.8230	0.8856
0.0049	46.0	1840	0.9860	0.8278	0.8329	0.8304	0.8879
0.0049	47.0	1880	0.9870	0.8278	0.8329	0.8304	0.8879
0.0049	48.0	1920	0.9896	0.8278	0.8329	0.8304	0.8879
0.0049	49.0	1960	0.9997	0.8230	0.8280	0.8255	0.8850
0.0022	50.0	2000	1.0007	0.8230	0.8280	0.8255	0.8850

Framework versions

• Transformers 4.34.1
• Pytorch 2.2.1+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1