results_1027_all

This model is a fine-tuned version of google/gemma-2b-it on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0101

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 3
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 24
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
3.6243	0.0390	10	4.0672
3.6311	0.0780	20	4.0332
3.7887	0.1170	30	3.9444
3.9806	0.1559	40	3.7516
5.4583	0.1949	50	3.4772
3.1139	0.2339	60	3.2387
2.9725	0.2729	70	3.0598
2.8967	0.3119	80	2.9309
2.8014	0.3509	90	2.8155
3.2525	0.3899	100	2.7300
2.7656	0.4288	110	2.6594
2.6288	0.4678	120	2.6049
2.5762	0.5068	130	2.5462
2.4191	0.5458	140	2.5070
2.3597	0.5848	150	2.4805
2.6752	0.6238	160	2.4712
2.536	0.6628	170	2.4236
2.4212	0.7018	180	2.3911
2.2347	0.7407	190	2.3861
1.9591	0.7797	200	2.3951
2.6298	0.8187	210	2.3823
2.496	0.8577	220	2.3475
2.3756	0.8967	230	2.3250
2.1597	0.9357	240	2.3126
1.7922	0.9747	250	2.3313
2.3355	1.0136	260	2.2854
2.4845	1.0526	270	2.2925
2.2919	1.0916	280	2.2642
2.2167	1.1306	290	2.2538
1.9066	1.1696	300	2.2615
1.8711	1.2086	310	2.2753
2.4956	1.2476	320	2.2714
2.2881	1.2865	330	2.2358
2.2281	1.3255	340	2.2230
1.9285	1.3645	350	2.2265
1.8699	1.4035	360	2.2408
2.511	1.4425	370	2.2339
2.2433	1.4815	380	2.2024
2.1443	1.5205	390	2.1929
1.8879	1.5595	400	2.1998
1.8077	1.5984	410	2.2197
2.5475	1.6374	420	2.2101
2.2301	1.6764	430	2.1825
2.189	1.7154	440	2.1749
1.8535	1.7544	450	2.1760
1.7574	1.7934	460	2.1864
2.4951	1.8324	470	2.1951
2.2221	1.8713	480	2.1625
2.1631	1.9103	490	2.1567
1.804	1.9493	500	2.1594
1.7228	1.9883	510	2.1651
2.2207	2.0273	520	2.1558
2.3622	2.0663	530	2.1531
2.1465	2.1053	540	2.1417
2.0108	2.1442	550	2.1439
1.5252	2.1832	560	2.1619
2.0833	2.2222	570	2.1479
2.3643	2.2612	580	2.1390
2.1568	2.3002	590	2.1281
2.0213	2.3392	600	2.1275
1.5785	2.3782	610	2.1321
2.0772	2.4172	620	2.1304
2.349	2.4561	630	2.1236
2.1461	2.4951	640	2.1167
2.0111	2.5341	650	2.1173
1.492	2.5731	660	2.1310
2.0511	2.6121	670	2.1287
2.3421	2.6511	680	2.1157
2.1066	2.6901	690	2.1082
2.0321	2.7290	700	2.1090
1.5423	2.7680	710	2.1182
2.0467	2.8070	720	2.1125
2.3023	2.8460	730	2.1039
2.1631	2.8850	740	2.0987
2.0039	2.9240	750	2.0980
1.5499	2.9630	760	2.1154
1.759	3.0019	770	2.1043
2.4151	3.0409	780	2.0976
2.1606	3.0799	790	2.0898
2.0559	3.1189	800	2.0871
1.838	3.1579	810	2.0947
1.326	3.1969	820	2.1144
2.3495	3.2359	830	2.1006
2.2127	3.2749	840	2.0876
2.1038	3.3138	850	2.0839
1.854	3.3528	860	2.0878
1.3388	3.3918	870	2.1050
2.3707	3.4308	880	2.0895
2.2118	3.4698	890	2.0812
2.0794	3.5088	900	2.0776
1.8689	3.5478	910	2.0775
1.3517	3.5867	920	2.0923
2.3716	3.6257	930	2.0781
2.1935	3.6647	940	2.0716
2.0464	3.7037	950	2.0696
1.8611	3.7427	960	2.0732
1.3304	3.7817	970	2.0892
2.3685	3.8207	980	2.0788
2.1751	3.8596	990	2.0717
2.0609	3.8986	1000	2.0708
1.8034	3.9376	1010	2.0707
1.2843	3.9766	1020	2.0796
2.0898	4.0156	1030	2.0689
2.2856	4.0546	1040	2.0657
2.0634	4.0936	1050	2.0605
1.9761	4.1326	1060	2.0623
1.6156	4.1715	1070	2.0712
1.613	4.2105	1080	2.0718
2.3393	4.2495	1090	2.0740
2.0386	4.2885	1100	2.0584
1.9765	4.3275	1110	2.0585
1.6274	4.3665	1120	2.0709
1.6481	4.4055	1130	2.0712
2.3503	4.4444	1140	2.0654
2.0246	4.4834	1150	2.0520
1.9933	4.5224	1160	2.0531
1.5915	4.5614	1170	2.0610
1.6134	4.6004	1180	2.0598
2.3301	4.6394	1190	2.0581
2.0412	4.6784	1200	2.0474
1.9914	4.7173	1210	2.0495
1.6165	4.7563	1220	2.0519
1.6165	4.7953	1230	2.0542
2.2896	4.8343	1240	2.0539
2.0835	4.8733	1250	2.0464
2.0175	4.9123	1260	2.0478
1.6252	4.9513	1270	2.0572
1.5814	4.9903	1280	2.0508
2.082	5.0292	1290	2.0545
2.1944	5.0682	1300	2.0406
1.9964	5.1072	1310	2.0408
1.8602	5.1462	1320	2.0431
1.3353	5.1852	1330	2.0630
1.991	5.2242	1340	2.0474
2.22	5.2632	1350	2.0468
2.033	5.3021	1360	2.0392
1.9036	5.3411	1370	2.0433
1.2878	5.3801	1380	2.0563
1.9576	5.4191	1390	2.0397
2.2161	5.4581	1400	2.0423
2.0268	5.4971	1410	2.0345
1.8721	5.5361	1420	2.0368
1.3535	5.5750	1430	2.0453
2.0218	5.6140	1440	2.0377
2.1735	5.6530	1450	2.0361
1.992	5.6920	1460	2.0313
1.8533	5.7310	1470	2.0364
1.3057	5.7700	1480	2.0497
1.9589	5.8090	1490	2.0371
2.2148	5.8480	1500	2.0326
2.0038	5.8869	1510	2.0278
1.8754	5.9259	1520	2.0326
1.4612	5.9649	1530	2.0367
1.734	6.0039	1540	2.0317
2.2545	6.0429	1550	2.0409
2.0817	6.0819	1560	2.0253
1.9647	6.1209	1570	2.0269
1.7068	6.1598	1580	2.0348
1.2406	6.1988	1590	2.0481
2.3035	6.2378	1600	2.0275
2.0767	6.2768	1610	2.0282
1.963	6.3158	1620	2.0271
1.7227	6.3548	1630	2.0364
1.2378	6.3938	1640	2.0472
2.2687	6.4327	1650	2.0258
2.0813	6.4717	1660	2.0273
1.9627	6.5107	1670	2.0223
1.7152	6.5497	1680	2.0299
1.2746	6.5887	1690	2.0358
2.2563	6.6277	1700	2.0222
2.0765	6.6667	1710	2.0241
1.946	6.7057	1720	2.0213
1.7193	6.7446	1730	2.0248
1.2454	6.7836	1740	2.0417
2.2963	6.8226	1750	2.0224
2.0297	6.8616	1760	2.0220
1.9472	6.9006	1770	2.0191
1.7076	6.9396	1780	2.0240
1.263	6.9786	1790	2.0336
2.016	7.0175	1800	2.0185
2.2479	7.0565	1810	2.0205
1.9657	7.0955	1820	2.0173
1.8752	7.1345	1830	2.0234
1.518	7.1735	1840	2.0280
1.5781	7.2125	1850	2.0280
2.2415	7.2515	1860	2.0171
1.95	7.2904	1870	2.0176
1.8691	7.3294	1880	2.0181
1.4526	7.3684	1890	2.0304
1.5953	7.4074	1900	2.0284
2.2171	7.4464	1910	2.0150
1.9853	7.4854	1920	2.0163
1.8982	7.5244	1930	2.0148
1.511	7.5634	1940	2.0275
1.6055	7.6023	1950	2.0265
2.1793	7.6413	1960	2.0140
1.9537	7.6803	1970	2.0139
1.8968	7.7193	1980	2.0157
1.4738	7.7583	1990	2.0219
1.5763	7.7973	2000	2.0193
2.1311	7.8363	2010	2.0104
1.9862	7.8752	2020	2.0126
1.8722	7.9142	2030	2.0129
1.4885	7.9532	2040	2.0174
1.4697	7.9922	2050	2.0173
2.0672	8.0312	2060	2.0111
2.0969	8.0702	2070	2.0108
1.8989	8.1092	2080	2.0127
1.7529	8.1481	2090	2.0198
1.21	8.1871	2100	2.0289
1.9496	8.2261	2110	2.0142
2.1324	8.2651	2120	2.0123
1.944	8.3041	2130	2.0115
1.7891	8.3431	2140	2.0158
1.2247	8.3821	2150	2.0350
1.9617	8.4211	2160	2.0138
2.0824	8.4600	2170	2.0118
1.949	8.4990	2180	2.0111
1.778	8.5380	2190	2.0125
1.2231	8.5770	2200	2.0229
1.9759	8.6160	2210	2.0144
2.1415	8.6550	2220	2.0112
1.9389	8.6940	2230	2.0121
1.7935	8.7329	2240	2.0128
1.2627	8.7719	2250	2.0201
1.9485	8.8109	2260	2.0086
2.1153	8.8499	2270	2.0080
1.9359	8.8889	2280	2.0065
1.7867	8.9279	2290	2.0121
1.2151	8.9669	2300	2.0238
1.6928	9.0058	2310	2.0126
2.2037	9.0448	2320	2.0084
1.99	9.0838	2330	2.0079
1.8715	9.1228	2340	2.0091
1.5954	9.1618	2350	2.0195
1.2096	9.2008	2360	2.0270
2.1857	9.2398	2370	2.0069
2.055	9.2788	2380	2.0088
1.9266	9.3177	2390	2.0093
1.5987	9.3567	2400	2.0173
1.2183	9.3957	2410	2.0255
2.1812	9.4347	2420	2.0076
1.9888	9.4737	2430	2.0101
1.8815	9.5127	2440	2.0090
1.6215	9.5517	2450	2.0144
1.2026	9.5906	2460	2.0247
2.1664	9.6296	2470	2.0065
2.0357	9.6686	2480	2.0089
1.867	9.7076	2490	2.0080
1.6387	9.7466	2500	2.0141
1.2534	9.7856	2510	2.0213
2.2086	9.8246	2520	2.0055
1.9423	9.8635	2530	2.0061
1.8702	9.9025	2540	2.0056
1.6139	9.9415	2550	2.0137
1.2111	9.9805	2560	2.0172
1.9268	10.0195	2570	2.0052
2.1165	10.0585	2580	2.0060
1.9097	10.0975	2590	2.0055
1.8199	10.1365	2600	2.0103
1.3818	10.1754	2610	2.0213
1.5802	10.2144	2620	2.0201
2.1623	10.2534	2630	2.0046
1.9096	10.2924	2640	2.0101
1.7922	10.3314	2650	2.0117
1.3309	10.3704	2660	2.0191
1.5497	10.4094	2670	2.0124
2.1491	10.4483	2680	2.0037
1.9029	10.4873	2690	2.0073
1.8312	10.5263	2700	2.0077
1.3983	10.5653	2710	2.0179
1.5484	10.6043	2720	2.0156
2.1516	10.6433	2730	2.0051
1.909	10.6823	2740	2.0070
1.7929	10.7212	2750	2.0067
1.3847	10.7602	2760	2.0138
1.5824	10.7992	2770	2.0099
2.1721	10.8382	2780	2.0024
1.9055	10.8772	2790	2.0055
1.8087	10.9162	2800	2.0078
1.428	10.9552	2810	2.0140
1.4807	10.9942	2820	2.0145
2.0324	11.0331	2830	2.0029
2.0896	11.0721	2840	2.0046
1.9081	11.1111	2850	2.0061
1.717	11.1501	2860	2.0115
1.164	11.1891	2870	2.0244
1.9655	11.2281	2880	2.0086
2.0216	11.2671	2890	2.0038
1.8918	11.3060	2900	2.0090
1.7195	11.3450	2910	2.0124
1.1172	11.3840	2920	2.0194
1.9826	11.4230	2930	2.0097
2.0471	11.4620	2940	2.0040
1.8715	11.5010	2950	2.0093
1.6894	11.5400	2960	2.0125
1.1379	11.5789	2970	2.0181
1.9571	11.6179	2980	2.0078
2.0021	11.6569	2990	2.0013
1.853	11.6959	3000	2.0054
1.7275	11.7349	3010	2.0090
1.1456	11.7739	3020	2.0157
1.9835	11.8129	3030	2.0071
2.0775	11.8519	3040	2.0018
1.8909	11.8908	3050	2.0062
1.6822	11.9298	3060	2.0091
1.1052	11.9688	3070	2.0150
1.6883	12.0078	3080	2.0096
2.1567	12.0468	3090	2.0041
1.9456	12.0858	3100	2.0047
1.8427	12.1248	3110	2.0069
1.6165	12.1637	3120	2.0121
1.2291	12.2027	3130	2.0214
2.1499	12.2417	3140	2.0069
1.9792	12.2807	3150	2.0037
1.8188	12.3197	3160	2.0092
1.5767	12.3587	3170	2.0151
1.1964	12.3977	3180	2.0210
2.1613	12.4366	3190	2.0044
1.9354	12.4756	3200	2.0026
1.8275	12.5146	3210	2.0083
1.5021	12.5536	3220	2.0136
1.2069	12.5926	3230	2.0196
2.123	12.6316	3240	2.0057
1.9439	12.6706	3250	2.0034
1.8142	12.7096	3260	2.0082
1.514	12.7485	3270	2.0143
1.1998	12.7875	3280	2.0196
2.1347	12.8265	3290	2.0051
1.9141	12.8655	3300	2.0007
1.8228	12.9045	3310	2.0058
1.5005	12.9435	3320	2.0112
1.2364	12.9825	3330	2.0178
1.9069	13.0214	3340	2.0060
2.1331	13.0604	3350	2.0045
1.8384	13.0994	3360	2.0043
1.7817	13.1384	3370	2.0089
1.3311	13.1774	3380	2.0167
1.5687	13.2164	3390	2.0169
2.1207	13.2554	3400	2.0040
1.8559	13.2943	3410	2.0037
1.7614	13.3333	3420	2.0101
1.3	13.3723	3430	2.0173
1.559	13.4113	3440	2.0173
2.0876	13.4503	3450	2.0041
1.8945	13.4893	3460	2.0037
1.761	13.5283	3470	2.0088
1.2769	13.5673	3480	2.0141
1.5452	13.6062	3490	2.0158
2.1261	13.6452	3500	2.0054
1.8561	13.6842	3510	2.0028
1.7431	13.7232	3520	2.0083
1.2973	13.7622	3530	2.0149
1.5876	13.8012	3540	2.0145
2.066	13.8402	3550	2.0037
1.8892	13.8791	3560	2.0035
1.7726	13.9181	3570	2.0076
1.353	13.9571	3580	2.0126
1.4365	13.9961	3590	2.0123
1.991	14.0351	3600	2.0056
2.0209	14.0741	3610	2.0034
1.8281	14.1131	3620	2.0057
1.6186	14.1520	3630	2.0097
1.0649	14.1910	3640	2.0179
2.0154	14.2300	3650	2.0144
2.0405	14.2690	3660	2.0063
1.8228	14.3080	3670	2.0057
1.6807	14.3470	3680	2.0122
1.096	14.3860	3690	2.0172
2.0296	14.4250	3700	2.0102
1.9521	14.4639	3710	2.0035
1.8553	14.5029	3720	2.0037
1.6367	14.5419	3730	2.0089
1.0345	14.5809	3740	2.0154
1.9927	14.6199	3750	2.0126
1.9761	14.6589	3760	2.0054
1.8645	14.6979	3770	2.0050
1.6667	14.7368	3780	2.0093
1.0332	14.7758	3790	2.0125
2.0177	14.8148	3800	2.0082
2.0033	14.8538	3810	2.0036
1.8773	14.8928	3820	2.0027
1.683	14.9318	3830	2.0061
1.0772	14.9708	3840	2.0126
1.7342	15.0097	3850	2.0129
2.1226	15.0487	3860	2.0072
1.8864	15.0877	3870	2.0041
1.8082	15.1267	3880	2.0059
1.4591	15.1657	3890	2.0106
1.211	15.2047	3900	2.0152
2.1329	15.2437	3910	2.0106
1.8654	15.2827	3920	2.0059
1.7968	15.3216	3930	2.0066
1.4026	15.3606	3940	2.0107
1.2165	15.3996	3950	2.0132
2.1389	15.4386	3960	2.0091
1.8987	15.4776	3970	2.0063
1.8091	15.5166	3980	2.0070
1.4927	15.5556	3990	2.0103
1.2281	15.5945	4000	2.0140
2.1384	15.6335	4010	2.0110
1.9236	15.6725	4020	2.0073
1.7726	15.7115	4030	2.0066
1.5545	15.7505	4040	2.0093
1.2426	15.7895	4050	2.0122
2.1387	15.8285	4060	2.0096
1.9333	15.8674	4070	2.0062
1.8129	15.9064	4080	2.0059
1.4992	15.9454	4090	2.0081
1.2263	15.9844	4100	2.0104
1.8705	16.0234	4110	2.0095
2.0751	16.0624	4120	2.0074
1.8635	16.1014	4130	2.0060
1.7449	16.1404	4140	2.0075
1.2633	16.1793	4150	2.0105
1.6145	16.2183	4160	2.0127
2.0626	16.2573	4170	2.0102
1.8527	16.2963	4180	2.0082
1.746	16.3353	4190	2.0085
1.2795	16.3743	4200	2.0105
1.5979	16.4133	4210	2.0115
2.0834	16.4522	4220	2.0091
1.8414	16.4912	4230	2.0070
1.7203	16.5302	4240	2.0079
1.2432	16.5692	4250	2.0098
1.6078	16.6082	4260	2.0110
2.0594	16.6472	4270	2.0096
1.8578	16.6862	4280	2.0082
1.6966	16.7251	4290	2.0084
1.2712	16.7641	4300	2.0101
1.6218	16.8031	4310	2.0114
2.0752	16.8421	4320	2.0100
1.823	16.8811	4330	2.0084
1.7448	16.9201	4340	2.0085
1.2012	16.9591	4350	2.0094
1.4111	16.9981	4360	2.0102
2.0496	17.0370	4370	2.0098
1.9603	17.0760	4380	2.0088
1.8471	17.1150	4390	2.0084
1.6178	17.1540	4400	2.0099
1.0203	17.1930	4410	2.0114
2.1002	17.2320	4420	2.0112
1.9752	17.2710	4430	2.0098
1.8138	17.3099	4440	2.0092
1.629	17.3489	4450	2.0096
1.0108	17.3879	4460	2.0106
2.024	17.4269	4470	2.0107
1.9937	17.4659	4480	2.0096
1.8148	17.5049	4490	2.0089
1.6422	17.5439	4500	2.0093
0.9604	17.5828	4510	2.0103
2.0572	17.6218	4520	2.0105
1.9585	17.6608	4530	2.0097
1.8546	17.6998	4540	2.0091
1.6058	17.7388	4550	2.0096
1.0228	17.7778	4560	2.0106
2.0638	17.8168	4570	2.0107
1.9718	17.8558	4580	2.0101
1.8256	17.8947	4590	2.0094
1.6277	17.9337	4600	2.0095
1.0375	17.9727	4610	2.0101
1.7781	18.0117	4620	2.0106
2.0951	18.0507	4630	2.0102
1.8708	18.0897	4640	2.0096
1.7746	18.1287	4650	2.0093
1.5046	18.1676	4660	2.0098
1.2636	18.2066	4670	2.0102
2.0885	18.2456	4680	2.0101
1.9133	18.2846	4690	2.0096
1.7877	18.3236	4700	2.0094
1.4113	18.3626	4710	2.0097
1.2654	18.4016	4720	2.0102
2.1448	18.4405	4730	2.0102
1.8501	18.4795	4740	2.0100
1.8131	18.5185	4750	2.0099
1.4439	18.5575	4760	2.0102
1.2743	18.5965	4770	2.0104
2.1222	18.6355	4780	2.0103
1.8439	18.6745	4790	2.0100
1.7569	18.7135	4800	2.0099
1.4505	18.7524	4810	2.0101
1.2645	18.7914	4820	2.0103
2.1523	18.8304	4830	2.0102
1.9206	18.8694	4840	2.0099
1.774	18.9084	4850	2.0098
1.3784	18.9474	4860	2.0099
1.2127	18.9864	4870	2.0101
1.888	19.0253	4880	2.0102
2.0638	19.0643	4890	2.0101
1.8331	19.1033	4900	2.0100
1.7499	19.1423	4910	2.0100
1.2186	19.1813	4920	2.0101
1.6837	19.2203	4930	2.0101
2.0393	19.2593	4940	2.0101
1.8557	19.2982	4950	2.0100
1.7014	19.3372	4960	2.0100
1.1426	19.3762	4970	2.0101
1.6674	19.4152	4980	2.0102
2.0397	19.4542	4990	2.0101
1.8376	19.4932	5000	2.0101
1.7296	19.5322	5010	2.0101
1.217	19.5712	5020	2.0101
1.6529	19.6101	5030	2.0101
2.0351	19.6491	5040	2.0101
1.8406	19.6881	5050	2.0101
1.6803	19.7271	5060	2.0101
1.1903	19.7661	5070	2.0101
1.726	19.8051	5080	2.0101
2.0515	19.8441	5090	2.0101
1.8248	19.8830	5100	2.0101
1.7132	19.9220	5110	2.0101
1.259	19.9610	5120	2.0101

Framework versions

• PEFT 0.12.0
• Transformers 4.45.0
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.20.1