qwenvl-2B-cadica-direction-scale4

This model is a fine-tuned version of AdaptLLM/biomed-Qwen2-VL-2B-Instruct on the CADICA血管分支方向題scale4(TRAIN) dataset. It achieves the following results on the evaluation set:

• Loss: 0.0000
• Num Input Tokens Seen: 11990784

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 6
• total_train_batch_size: 24
• total_eval_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• training_steps: 1200

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3441	0.0258	50	0.3383	499200
0.2274	0.0515	100	0.1866	998400
0.0667	0.0773	150	0.0967	1497600
0.0459	0.1030	200	0.0996	1996800
0.0805	0.1288	250	0.0559	2496000
0.0381	0.1545	300	0.0309	2995200
0.1761	0.1803	350	0.0439	3494400
0.0146	0.2060	400	0.0244	3993600
0.0157	0.2318	450	0.0067	4492800
0.0122	0.2575	500	0.0080	4992000
0.0339	0.2833	550	0.0034	5491200
0.0217	0.3090	600	0.0133	5990400
0.0327	0.3348	650	0.0210	6489600
0.0267	0.3605	700	0.0053	6988800
0.014	0.3863	750	0.0053	7488000
0.0065	0.4121	800	0.0068	7987200
0.0306	0.4378	850	0.0072	8486400
0.0063	0.4636	900	0.0107	8985600
0.0415	0.4893	950	0.0072	9484800
0.0547	0.5151	1000	0.0007	9984000
0.0007	0.5408	1050	0.0568	10483200
0.0056	0.5666	1100	0.0004	10982400
0.0127	0.5923	1150	0.0000	11481600
0.0038	0.6181	1200	0.0022	11980800

Framework versions

• PEFT 0.12.0
• Transformers 4.47.0.dev0
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3