Mists-7B-v01-projector-trained

This model is a fine-tuned version of HachiML/Mists-7B-v01-not-trained on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5989

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.002
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.8704	0.0526	2000	0.7797
0.7321	0.1051	4000	0.7205
0.7002	0.1577	6000	0.6992
0.6735	0.2102	8000	0.6755
0.6644	0.2628	10000	0.6625
0.6562	0.3153	12000	0.6539
0.6512	0.3679	14000	0.6473
0.638	0.4205	16000	0.6394
0.6266	0.4730	18000	0.6325
0.627	0.5256	20000	0.6278
0.6215	0.5781	22000	0.6241
0.6192	0.6307	24000	0.6199
0.6127	0.6832	26000	0.6163
0.6121	0.7358	28000	0.6131
0.6078	0.7884	30000	0.6089
0.5986	0.8409	32000	0.6058
0.6011	0.8935	34000	0.6027
0.5934	0.9460	36000	0.6005
0.589	0.9986	38000	0.5989

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1