swiftformer-xs-ve-U13-b-80

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.7132
• Accuracy: 0.8261

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	6	1.3859	0.2391
1.3857	2.0	13	1.3834	0.2826
1.3857	2.92	19	1.3789	0.1957
1.3767	4.0	26	1.3666	0.1522
1.3226	4.92	32	1.3565	0.1522
1.3226	6.0	39	1.3902	0.1087
1.1987	6.92	45	1.3712	0.2174
1.1075	8.0	52	1.3197	0.3478
1.1075	8.92	58	1.3649	0.3696
0.9988	10.0	65	1.2583	0.3696
0.8863	10.92	71	1.2484	0.3696
0.8863	12.0	78	1.2869	0.4130
0.8228	12.92	84	1.1678	0.4783
0.7456	14.0	91	1.0275	0.6739
0.7456	14.92	97	0.9702	0.7174
0.6595	16.0	104	0.9103	0.6957
0.5995	16.92	110	0.8506	0.7391
0.5995	18.0	117	0.8514	0.7174
0.5826	18.92	123	0.8964	0.7391
0.4818	20.0	130	0.8550	0.7609
0.4818	20.92	136	0.7132	0.8261
0.4553	22.0	143	0.6973	0.7826
0.4553	22.92	149	0.7496	0.7391
0.4276	24.0	156	0.9087	0.6957
0.3375	24.92	162	0.7787	0.8261
0.3375	26.0	169	0.7132	0.8043
0.3199	26.92	175	0.7570	0.7391
0.2756	28.0	182	0.7873	0.6957
0.2756	28.92	188	0.7895	0.7609
0.2254	30.0	195	0.7443	0.8043
0.2576	30.92	201	0.9623	0.6739
0.2576	32.0	208	0.7349	0.7826
0.2113	32.92	214	0.7887	0.7609
0.1978	34.0	221	0.8921	0.7391
0.1978	34.92	227	0.8102	0.7391
0.2455	36.0	234	0.8947	0.7391
0.1809	36.92	240	0.8144	0.7826
0.1809	38.0	247	0.8290	0.7174
0.1967	38.92	253	0.8135	0.7391
0.1608	40.0	260	0.8065	0.7609
0.1608	40.92	266	0.7399	0.7609
0.1704	42.0	273	0.7099	0.8043
0.1704	42.92	279	0.7569	0.7826
0.1682	44.0	286	0.8459	0.7826
0.1607	44.92	292	0.7311	0.7609
0.1607	46.0	299	0.7833	0.7174
0.1589	46.92	305	0.8073	0.6957
0.1524	48.0	312	0.7473	0.7609
0.1524	48.92	318	0.6780	0.8043
0.1586	50.0	325	0.7573	0.7174
0.128	50.92	331	0.7614	0.7391
0.128	52.0	338	0.7338	0.7609
0.1254	52.92	344	0.7666	0.7391
0.1206	54.0	351	0.8433	0.7174
0.1206	54.92	357	0.8747	0.6957
0.1398	56.0	364	0.8940	0.7174
0.1536	56.92	370	0.7781	0.7826
0.1536	58.0	377	0.7351	0.7391
0.1281	58.92	383	0.7601	0.7174
0.1156	60.0	390	0.7991	0.7174
0.1156	60.92	396	0.7776	0.7609
0.0852	62.0	403	0.7838	0.7391
0.0852	62.92	409	0.7752	0.7609
0.1106	64.0	416	0.7541	0.7609
0.0817	64.92	422	0.7536	0.7391
0.0817	66.0	429	0.8129	0.7609
0.1211	66.92	435	0.7884	0.7609
0.0944	68.0	442	0.8011	0.7609
0.0944	68.92	448	0.8068	0.7391
0.1187	70.0	455	0.7796	0.7391
0.0935	70.92	461	0.7934	0.7391
0.0935	72.0	468	0.7367	0.7391
0.109	72.92	474	0.7515	0.7391
0.1006	73.85	480	0.7888	0.7174

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0