vit-base-patch16-224-dmae-va-U

This model is a fine-tuned version of google/vit-base-patch16-224 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0534
• Accuracy: 0.9908

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.9	7	1.4319	0.2569
1.3911	1.94	15	1.2133	0.4771
1.3911	2.97	23	0.9487	0.6055
1.0766	4.0	31	0.6542	0.7156
0.6974	4.9	38	0.4644	0.8716
0.6974	5.94	46	0.3919	0.8716
0.421	6.97	54	0.3094	0.8716
0.2513	8.0	62	0.2334	0.8991
0.2513	8.9	69	0.1915	0.9174
0.1931	9.94	77	0.2431	0.8807
0.1757	10.97	85	0.1608	0.9450
0.1757	12.0	93	0.1424	0.9266
0.1442	12.9	100	0.1280	0.9450
0.1085	13.94	108	0.1055	0.9541
0.1085	14.97	116	0.1080	0.9541
0.1056	16.0	124	0.0997	0.9633
0.1056	16.9	131	0.1185	0.9633
0.0926	17.94	139	0.0773	0.9633
0.103	18.97	147	0.1279	0.9633
0.103	20.0	155	0.1043	0.9633
0.0938	20.9	162	0.0824	0.9817
0.0891	21.94	170	0.1449	0.9541
0.0891	22.97	178	0.1366	0.9633
0.0754	24.0	186	0.1148	0.9358
0.0882	24.9	193	0.1992	0.9358
0.0882	25.94	201	0.0743	0.9817
0.078	26.97	209	0.0668	0.9725
0.0666	28.0	217	0.0534	0.9908
0.0666	28.9	224	0.0499	0.9908
0.0514	29.94	232	0.0433	0.9725
0.062	30.97	240	0.0840	0.9633
0.062	32.0	248	0.0513	0.9725
0.0712	32.9	255	0.0482	0.9817
0.0712	33.94	263	0.0553	0.9817
0.0703	34.97	271	0.0602	0.9725
0.0553	36.0	279	0.0595	0.9725
0.0553	36.13	280	0.0595	0.9725

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0