git-base-lora-finetune

This model is a fine-tuned version of microsoft/git-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 9.3318
• Wer Score: 66.9677

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 250
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
11.6181	9.0909	50	10.9172	70.8774
10.3832	18.1818	100	10.0450	59.4903
9.8974	27.2727	150	9.6934	82.2516
9.6607	36.3636	200	9.5085	77.4065
9.5472	45.4545	250	9.4289	72.5419
9.4973	54.5455	300	9.3941	72.3742
9.4753	63.6364	350	9.3773	71.8903
9.4632	72.7273	400	9.3670	70.8774
9.4554	81.8182	450	9.3602	70.2968
9.4496	90.9091	500	9.3550	70.0258
9.4453	100.0	550	9.3516	68.5419
9.4415	109.0909	600	9.3473	68.6065
9.4386	118.1818	650	9.3446	68.4065
9.4362	127.2727	700	9.3422	67.7548
9.434	136.3636	750	9.3403	67.6065
9.4324	145.4545	800	9.3379	67.6903
9.4306	154.5455	850	9.3370	68.8387
9.4296	163.6364	900	9.3359	67.4
9.4284	172.7273	950	9.3350	67.6645
9.4276	181.8182	1000	9.3342	67.5613
9.427	190.9091	1050	9.3333	67.2581
9.4263	200.0	1100	9.3327	67.7484
9.4258	209.0909	1150	9.3322	67.0387
9.4256	218.1818	1200	9.3320	67.1677
9.4256	227.2727	1250	9.3318	66.9677

Framework versions

• PEFT 0.13.2
• Transformers 4.46.2
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3