git-base-pokemon

This model is a fine-tuned version of microsoft/git-base on the imagefolder dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0429
  • Wer Score: 1.9591

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Score
7.3666 1.06 50 4.4430 21.5287
2.1581 2.13 100 0.2911 0.9783
0.0896 3.19 150 0.0328 0.3665
0.0269 4.26 200 0.0274 0.3487
0.0208 5.32 250 0.0284 0.4189
0.0168 6.38 300 0.0287 1.1673
0.0133 7.45 350 0.0296 6.0881
0.0106 8.51 400 0.0306 1.7969
0.0076 9.57 450 0.0322 7.1852
0.0053 10.64 500 0.0329 14.8889
0.0039 11.7 550 0.0338 12.2720
0.0027 12.77 600 0.0356 5.1533
0.0016 13.83 650 0.0371 8.4253
0.001 14.89 700 0.0379 6.7344
0.0006 15.96 750 0.0385 7.7586
0.0005 17.02 800 0.0392 9.0294
0.0004 18.09 850 0.0385 7.5083
0.0004 19.15 900 0.0394 5.1188
0.0004 20.21 950 0.0397 5.0600
0.0004 21.28 1000 0.0399 4.4125
0.0003 22.34 1050 0.0405 3.7803
0.0003 23.4 1100 0.0406 3.3397
0.0003 24.47 1150 0.0408 3.3218
0.0003 25.53 1200 0.0411 2.8212
0.0003 26.6 1250 0.0411 2.7165
0.0003 27.66 1300 0.0414 2.7625
0.0003 28.72 1350 0.0416 2.4330
0.0003 29.79 1400 0.0416 2.2350
0.0003 30.85 1450 0.0419 2.1699
0.0003 31.91 1500 0.0421 2.0026
0.0003 32.98 1550 0.0420 2.1609
0.0003 34.04 1600 0.0421 2.0307
0.0003 35.11 1650 0.0422 1.9668
0.0003 36.17 1700 0.0423 1.9387
0.0003 37.23 1750 0.0425 1.9464
0.0003 38.3 1800 0.0427 1.8761
0.0003 39.36 1850 0.0427 1.8940
0.0003 40.43 1900 0.0428 1.9068
0.0003 41.49 1950 0.0428 1.8774
0.0003 42.55 2000 0.0429 1.8352
0.0002 43.62 2050 0.0428 2.0907
0.0002 44.68 2100 0.0429 2.0319
0.0002 45.74 2150 0.0429 2.0179
0.0002 46.81 2200 0.0429 1.9706
0.0002 47.87 2250 0.0429 1.9604
0.0002 48.94 2300 0.0429 1.9540
0.0002 50.0 2350 0.0429 1.9591

Framework versions

  • Transformers 4.28.0
  • Pytorch 2.0.0+cu118
  • Datasets 2.11.0
  • Tokenizers 0.13.3
Downloads last month
15
Inference API
Inference API (serverless) does not yet support transformers models for this pipeline type.