Llama-2-7b-chat-hf-finetune_90_10_SY

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2073

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: reduce_lr_on_plateau
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.5063	0.9954	217	0.7119
0.2604	1.9908	434	0.7809
0.2264	2.9862	651	0.8667
0.1271	3.9817	868	0.9213
0.0844	4.9771	1085	0.9696
0.0837	5.9725	1302	1.0040
0.0814	6.9679	1519	1.0156
0.0752	7.9633	1736	1.0749
0.0726	8.9587	1953	1.0817
0.0659	9.9541	2170	1.0906
0.0567	10.9495	2387	1.1188
0.0534	11.9450	2604	1.1181
0.054	12.9404	2821	1.1292
0.0513	13.9358	3038	1.1397
0.0519	14.9312	3255	1.1484
0.0477	15.9266	3472	1.1570
0.0438	16.9220	3689	1.1620
0.059	17.9174	3906	1.1683
0.057	18.9128	4123	1.1738
0.0562	19.9083	4340	1.1769
0.0564	20.9037	4557	1.1782
0.0554	21.8991	4774	1.1808
0.0557	22.8945	4991	1.1832
0.0534	23.8899	5208	1.1859
0.0531	24.8853	5425	1.1887
0.0517	25.8807	5642	1.1910
0.0519	26.8761	5859	1.1935
0.0533	27.8716	6076	1.1955
0.0523	28.8670	6293	1.1973
0.0511	29.8624	6510	1.1993
0.0525	30.8578	6727	1.2010
0.0513	31.8532	6944	1.2025
0.0506	32.8486	7161	1.2039
0.0509	33.8440	7378	1.2054
0.0502	34.8394	7595	1.2056
0.0496	35.8349	7812	1.2057
0.0497	36.8303	8029	1.2058
0.0494	37.8257	8246	1.2060
0.0503	38.8211	8463	1.2063
0.0495	39.8165	8680	1.2065
0.0499	40.8119	8897	1.2066
0.0496	41.8073	9114	1.2068
0.0487	42.8028	9331	1.2070
0.0497	43.7982	9548	1.2072
0.0481	44.7936	9765	1.2072
0.0482	45.7890	9982	1.2072
0.0477	46.7844	10199	1.2072
0.0484	47.7798	10416	1.2072
0.0488	48.7752	10633	1.2072
0.0476	49.7706	10850	1.2073

Framework versions

• PEFT 0.10.0
• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1