Mists-7B-v01-single-turn

This model is a fine-tuned version of HachiML/Mists-7B-v01-projector-trained on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4305

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.7872	0.0420	400	1.0558
0.7703	0.0841	800	0.8494
0.6751	0.1261	1200	0.7353
0.6225	0.1682	1600	0.6600
0.5783	0.2102	2000	0.6104
0.5582	0.2523	2400	0.5712
0.5334	0.2943	2800	0.5482
0.5182	0.3363	3200	0.5290
0.4979	0.3784	3600	0.5098
0.4848	0.4204	4000	0.4933
0.4745	0.4625	4400	0.4821
0.4637	0.5045	4800	0.4730
0.4601	0.5466	5200	0.4633
0.4552	0.5886	5600	0.4562
0.4486	0.6306	6000	0.4503
0.441	0.6727	6400	0.4449
0.4425	0.7147	6800	0.4398
0.4306	0.7568	7200	0.4372
0.4246	0.7988	7600	0.4344
0.4232	0.8409	8000	0.4325
0.4279	0.8829	8400	0.4315
0.4274	0.9250	8800	0.4307
0.4224	0.9670	9200	0.4305

Framework versions

• Transformers 4.42.3
• Pytorch 2.0.1
• Datasets 2.20.0
• Tokenizers 0.19.1