gutsartificial/Phi-3.5-mini-instruct-rag-score-generator-adapters

Phi-3.5-mini-instruct-2024-10-28_15-54-04

This model is a fine-tuned version of unsloth/Phi-3.5-mini-instruct on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6209

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.6797	0.2492	148	0.6636
0.6457	0.4983	296	0.6456
0.6394	0.7475	444	0.6378
0.6539	0.9966	592	0.6329
0.6116	1.2458	740	0.6299
0.617	1.4949	888	0.6284
0.5936	1.7441	1036	0.6254
0.5994	1.9933	1184	0.6231
0.6277	2.2424	1332	0.6226
0.6123	2.4916	1480	0.6217
0.6583	2.7407	1628	0.6210
0.5918	2.9899	1776	0.6209

Framework versions

• PEFT 0.13.2
• Transformers 4.45.2
• Pytorch 2.4.0+cu121
• Datasets 2.19.2
• Tokenizers 0.20.1