fine-tuned-roberta2-nosql-injection

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0134

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 500

Training results

Training Loss	Epoch	Step	Validation Loss
1.2572	1.0	158	0.2235
0.1144	2.0	316	0.0546
0.0873	3.0	474	0.0943
0.0488	4.0	632	0.0324
0.0453	5.0	790	0.0970
0.1358	6.0	948	0.0003
0.0208	7.0	1106	0.0025
0.0409	8.0	1264	0.0000
0.0489	9.0	1422	0.0505
0.0808	10.0	1580	0.0000
0.0219	11.0	1738	0.0929
0.1204	12.0	1896	0.0000
0.0355	13.0	2054	0.0001
0.0093	14.0	2212	0.0671
0.0216	15.0	2370	0.0279
0.0388	16.0	2528	0.0173
0.0185	17.0	2686	0.0303
0.0069	18.0	2844	0.0160
0.0481	19.0	3002	0.0557
0.0459	20.0	3160	0.0022
0.0146	21.0	3318	0.0191
0.0313	22.0	3476	0.1119
0.025	23.0	3634	0.0000
0.0328	24.0	3792	0.0559
0.0251	25.0	3950	0.0343
0.0334	26.0	4108	0.0263
0.0048	27.0	4266	0.0424
0.0584	28.0	4424	0.0578
0.0861	29.0	4582	0.0495
0.0345	30.0	4740	0.0282
0.0193	31.0	4898	0.0412
0.0216	32.0	5056	0.0537
0.0241	33.0	5214	0.0124
0.009	34.0	5372	0.0272
0.0309	35.0	5530	0.0000
0.0069	36.0	5688	0.0000
0.0398	37.0	5846	0.0000
0.0474	38.0	6004	0.0017
0.0263	39.0	6162	0.0094
0.0128	40.0	6320	0.0000
0.0101	41.0	6478	0.0015
0.0415	42.0	6636	0.0004
0.0204	43.0	6794	0.0307
0.0342	44.0	6952	0.0348
0.0103	45.0	7110	0.0344
0.0389	46.0	7268	0.0238
0.0108	47.0	7426	0.0651
0.0486	48.0	7584	0.0290
0.0131	49.0	7742	0.0130
0.0027	50.0	7900	0.1448
0.0119	51.0	8058	0.0034
0.0142	52.0	8216	0.0014
0.0123	53.0	8374	0.0292
0.0364	54.0	8532	0.0000
0.0541	55.0	8690	0.0000
0.0309	56.0	8848	0.0032
0.0278	57.0	9006	0.0124
0.0247	58.0	9164	0.0000
0.0077	59.0	9322	0.0000
0.0104	60.0	9480	0.0150
0.0302	61.0	9638	0.0003
0.0225	62.0	9796	0.0358
0.0296	63.0	9954	0.0000
0.0533	64.0	10112	0.1224
0.0121	65.0	10270	0.0092
0.0249	66.0	10428	0.0087
0.0637	67.0	10586	0.0003
0.035	68.0	10744	0.0000
0.0117	69.0	10902	0.0393
0.0049	70.0	11060	0.0209
0.0225	71.0	11218	0.0003
0.0121	72.0	11376	0.0001
0.0079	73.0	11534	0.0006
0.0354	74.0	11692	0.0458
0.0071	75.0	11850	0.0009
0.0043	76.0	12008	0.0290
0.0239	77.0	12166	0.0001
0.0159	78.0	12324	0.0162
0.0051	79.0	12482	0.0673
0.0028	80.0	12640	0.0275
0.0155	81.0	12798	0.0101
0.0068	82.0	12956	0.0271
0.006	83.0	13114	0.0120
0.0107	84.0	13272	0.0008
0.0122	85.0	13430	0.0124
0.0028	86.0	13588	0.0000
0.0256	87.0	13746	0.0011
0.037	88.0	13904	0.0213
0.0044	89.0	14062	0.0000
0.003	90.0	14220	0.0000
0.0072	91.0	14378	0.0381
0.0309	92.0	14536	0.0000
0.0123	93.0	14694	0.0117
0.0002	94.0	14852	0.0024
0.0033	95.0	15010	0.0002
0.0181	96.0	15168	0.0071
0.0167	97.0	15326	0.0219
0.0136	98.0	15484	0.0001
0.0116	99.0	15642	0.0189
0.0118	100.0	15800	0.0000
0.0345	101.0	15958	0.0006
0.0032	102.0	16116	0.0355
0.0003	103.0	16274	0.0484
0.0281	104.0	16432	0.0000
0.029	105.0	16590	0.0319
0.006	106.0	16748	0.0016
0.0001	107.0	16906	0.0608
0.024	108.0	17064	0.0000
0.0187	109.0	17222	0.0000
0.0038	110.0	17380	0.0000
0.0046	111.0	17538	0.0045
0.0024	112.0	17696	0.0000
0.033	113.0	17854	0.0001
0.0048	114.0	18012	0.0511
0.0005	115.0	18170	0.0000
0.0166	116.0	18328	0.0000
0.0113	117.0	18486	0.0122
0.0043	118.0	18644	0.0122
0.0703	119.0	18802	0.0013
0.0051	120.0	18960	0.0009
0.048	121.0	19118	0.0082
0.0408	122.0	19276	0.0137
0.018	123.0	19434	0.0365
0.0125	124.0	19592	0.0111
0.0075	125.0	19750	0.0179
0.0379	126.0	19908	0.0000
0.0029	127.0	20066	0.0181
0.007	128.0	20224	0.0610
0.0069	129.0	20382	0.0203
0.0082	130.0	20540	0.0000
0.0136	131.0	20698	0.1019
0.0152	132.0	20856	0.0000
0.0078	133.0	21014	0.0089
0.0003	134.0	21172	0.0000
0.0069	135.0	21330	0.0278
0.0089	136.0	21488	0.0330
0.0642	137.0	21646	0.0251
0.0077	138.0	21804	0.0015
0.0062	139.0	21962	0.0218
0.0203	140.0	22120	0.0032
0.0173	141.0	22278	0.0000
0.0149	142.0	22436	0.0231
0.011	143.0	22594	0.0218
0.0142	144.0	22752	0.0068
0.0075	145.0	22910	0.0067
0.0021	146.0	23068	0.0172
0.009	147.0	23226	0.0134
0.0072	148.0	23384	0.0050
0.0297	149.0	23542	0.0162
0.0189	150.0	23700	0.0000
0.0065	151.0	23858	0.0006
0.0423	152.0	24016	0.0102
0.0001	153.0	24174	0.0147
0.0607	154.0	24332	0.0062
0.0247	155.0	24490	0.0000
0.0187	156.0	24648	0.0000
0.025	157.0	24806	0.0278
0.0069	158.0	24964	0.0002
0.0024	159.0	25122	0.0002
0.0194	160.0	25280	0.0000
0.0072	161.0	25438	0.0000
0.0552	162.0	25596	0.0304
0.0114	163.0	25754	0.0184
0.001	164.0	25912	0.0000
0.0076	165.0	26070	0.0004
0.0158	166.0	26228	0.0000
0.0017	167.0	26386	0.0145
0.0009	168.0	26544	0.0436
0.0199	169.0	26702	0.0147
0.0067	170.0	26860	0.0003
0.0082	171.0	27018	0.0151
0.0079	172.0	27176	0.0018
0.0016	173.0	27334	0.0002
0.0095	174.0	27492	0.0152
0.0166	175.0	27650	0.0000
0.0018	176.0	27808	0.0495
0.0025	177.0	27966	0.0824
0.01	178.0	28124	0.0000
0.0054	179.0	28282	0.0072
0.0082	180.0	28440	0.0019
0.0	181.0	28598	0.0131
0.0034	182.0	28756	0.0163
0.0083	183.0	28914	0.0175
0.0035	184.0	29072	0.0111
0.009	185.0	29230	0.0004
0.015	186.0	29388	0.0000
0.0011	187.0	29546	0.0182
0.0142	188.0	29704	0.0581
0.0048	189.0	29862	0.0002
0.0021	190.0	30020	0.0002
0.0	191.0	30178	0.0334
0.016	192.0	30336	0.0003
0.0109	193.0	30494	0.0000
0.0085	194.0	30652	0.0028
0.002	195.0	30810	0.0000
0.002	196.0	30968	0.0144
0.0142	197.0	31126	0.0000
0.0727	198.0	31284	0.0163
0.0238	199.0	31442	0.0000
0.0171	200.0	31600	0.0141
0.0158	201.0	31758	0.0000
0.0049	202.0	31916	0.0086
0.0016	203.0	32074	0.0000
0.0088	204.0	32232	0.0007
0.0048	205.0	32390	0.0164
0.0103	206.0	32548	0.0080
0.0039	207.0	32706	0.0441
0.0041	208.0	32864	0.0000
0.027	209.0	33022	0.0004
0.0229	210.0	33180	0.0149
0.01	211.0	33338	0.0154
0.0036	212.0	33496	0.0191
0.0036	213.0	33654	0.0491
0.0316	214.0	33812	0.0000
0.0014	215.0	33970	0.0198
0.0097	216.0	34128	0.0000
0.0011	217.0	34286	0.0001
0.0063	218.0	34444	0.0010
0.0143	219.0	34602	0.0243
0.0022	220.0	34760	0.0275
0.002	221.0	34918	0.0000
0.0095	222.0	35076	0.0119
0.0016	223.0	35234	0.0001
0.0165	224.0	35392	0.0000
0.0106	225.0	35550	0.0279
0.0316	226.0	35708	0.0006
0.0041	227.0	35866	0.0000
0.0492	228.0	36024	0.0123
0.0034	229.0	36182	0.0196
0.0035	230.0	36340	0.0113
0.0197	231.0	36498	0.0000
0.0234	232.0	36656	0.0026
0.0102	233.0	36814	0.0250
0.0113	234.0	36972	0.0001
0.0008	235.0	37130	0.0000
0.0032	236.0	37288	0.0001
0.1045	237.0	37446	0.0000
0.0028	238.0	37604	0.0143
0.0028	239.0	37762	0.0143
0.0002	240.0	37920	0.0171
0.0302	241.0	38078	0.0406
0.0046	242.0	38236	0.0273
0.0124	243.0	38394	0.0754
0.0055	244.0	38552	0.0396
0.0102	245.0	38710	0.0003
0.0139	246.0	38868	0.0201
0.0367	247.0	39026	0.0343
0.0216	248.0	39184	0.0015
0.0091	249.0	39342	0.0637
0.0115	250.0	39500	0.0053
0.0254	251.0	39658	0.0299
0.0202	252.0	39816	0.0011
0.0005	253.0	39974	0.0181
0.0024	254.0	40132	0.0223
0.0002	255.0	40290	0.0001
0.0102	256.0	40448	0.0593
0.0133	257.0	40606	0.0000
0.0051	258.0	40764	0.0001
0.0027	259.0	40922	0.0225
0.0053	260.0	41080	0.0000
0.0073	261.0	41238	0.0108
0.0033	262.0	41396	0.0118
0.0024	263.0	41554	0.0233
0.02	264.0	41712	0.0236
0.0078	265.0	41870	0.0180
0.0117	266.0	42028	0.0000
0.0156	267.0	42186	0.0000
0.0041	268.0	42344	0.0009
0.0003	269.0	42502	0.0108
0.0049	270.0	42660	0.0000
0.0021	271.0	42818	0.0174
0.0083	272.0	42976	0.0201
0.0002	273.0	43134	0.0371
0.0219	274.0	43292	0.0000
0.0014	275.0	43450	0.0156
0.0102	276.0	43608	0.0002
0.0311	277.0	43766	0.0008
0.0115	278.0	43924	0.0269
0.0122	279.0	44082	0.0131
0.0021	280.0	44240	0.0000
0.0032	281.0	44398	0.0157
0.0079	282.0	44556	0.0146
0.0344	283.0	44714	0.0142
0.0086	284.0	44872	0.0000
0.0045	285.0	45030	0.0011
0.0077	286.0	45188	0.0117
0.0026	287.0	45346	0.0006
0.0042	288.0	45504	0.0069
0.0528	289.0	45662	0.0199
0.0162	290.0	45820	0.0090
0.0598	291.0	45978	0.0000
0.0293	292.0	46136	0.0142
0.0021	293.0	46294	0.0330
0.0073	294.0	46452	0.0029
0.0006	295.0	46610	0.0007
0.0065	296.0	46768	0.0005
0.0075	297.0	46926	0.0371
0.0017	298.0	47084	0.0001
0.0023	299.0	47242	0.0442
0.0059	300.0	47400	0.0000
0.0041	301.0	47558	0.0198
0.0372	302.0	47716	0.0123
0.0004	303.0	47874	0.0177
0.0008	304.0	48032	0.0172
0.0002	305.0	48190	0.0005
0.0064	306.0	48348	0.0000
0.0116	307.0	48506	0.0142
0.0113	308.0	48664	0.0054
0.0046	309.0	48822	0.0090
0.0026	310.0	48980	0.0118
0.0051	311.0	49138	0.0012
0.0027	312.0	49296	0.0099
0.0031	313.0	49454	0.0104
0.0031	314.0	49612	0.0098
0.0085	315.0	49770	0.0115
0.0023	316.0	49928	0.0000
0.0143	317.0	50086	0.0000
0.0003	318.0	50244	0.0090
0.0179	319.0	50402	0.0124
0.0012	320.0	50560	0.0010
0.009	321.0	50718	0.0219
0.0082	322.0	50876	0.0102
0.0084	323.0	51034	0.0016
0.0029	324.0	51192	0.0227
0.0023	325.0	51350	0.0148
0.0022	326.0	51508	0.0157
0.0065	327.0	51666	0.0132
0.0257	328.0	51824	0.0162
0.0013	329.0	51982	0.0005
0.0135	330.0	52140	0.0278
0.0029	331.0	52298	0.0391
0.0139	332.0	52456	0.0000
0.0152	333.0	52614	0.0005
0.0042	334.0	52772	0.0008
0.0161	335.0	52930	0.0005
0.0008	336.0	53088	0.0000
0.0098	337.0	53246	0.0149
0.0096	338.0	53404	0.0097
0.0158	339.0	53562	0.0121
0.0001	340.0	53720	0.0168
0.0068	341.0	53878	0.0006
0.0026	342.0	54036	0.0000
0.0061	343.0	54194	0.0126
0.0071	344.0	54352	0.0154
0.0032	345.0	54510	0.0135
0.0034	346.0	54668	0.0046
0.003	347.0	54826	0.0137
0.0028	348.0	54984	0.0012
0.0157	349.0	55142	0.0193
0.0046	350.0	55300	0.0006
0.0023	351.0	55458	0.0137
0.0	352.0	55616	0.0111
0.0041	353.0	55774	0.0004
0.0032	354.0	55932	0.0000
0.0103	355.0	56090	0.0004
0.0	356.0	56248	0.0123
0.0074	357.0	56406	0.0205
0.0098	358.0	56564	0.0009
0.004	359.0	56722	0.0005
0.0018	360.0	56880	0.0134
0.0032	361.0	57038	0.0000
0.0063	362.0	57196	0.0230
0.0015	363.0	57354	0.0087
0.0041	364.0	57512	0.0110
0.0021	365.0	57670	0.0075
0.0019	366.0	57828	0.0000
0.0021	367.0	57986	0.0109
0.0018	368.0	58144	0.0228
0.0027	369.0	58302	0.0183
0.0036	370.0	58460	0.0116
0.0042	371.0	58618	0.0015
0.0029	372.0	58776	0.0010
0.0206	373.0	58934	0.0000
0.0071	374.0	59092	0.0000
0.0004	375.0	59250	0.0151
0.0009	376.0	59408	0.0000
0.0037	377.0	59566	0.0341
0.0097	378.0	59724	0.0239
0.0032	379.0	59882	0.0147
0.0041	380.0	60040	0.0006
0.002	381.0	60198	0.0003
0.0063	382.0	60356	0.0
0.0054	383.0	60514	0.0093
0.0069	384.0	60672	0.0089
0.0056	385.0	60830	0.0006
0.0071	386.0	60988	0.0219
0.0061	387.0	61146	0.0503
0.0068	388.0	61304	0.0028
0.002	389.0	61462	0.0011
0.0024	390.0	61620	0.0075
0.0028	391.0	61778	0.0009
0.0045	392.0	61936	0.0015
0.0009	393.0	62094	0.0350
0.0064	394.0	62252	0.0103
0.0003	395.0	62410	0.0118
0.0112	396.0	62568	0.0000
0.0046	397.0	62726	0.0118
0.0064	398.0	62884	0.0482
0.0021	399.0	63042	0.0228
0.0083	400.0	63200	0.0009
0.0008	401.0	63358	0.0005
0.0061	402.0	63516	0.0000
0.0038	403.0	63674	0.0031
0.0022	404.0	63832	0.0198
0.0074	405.0	63990	0.0000
0.0024	406.0	64148	0.0
0.0017	407.0	64306	0.0140
0.0004	408.0	64464	0.0210
0.003	409.0	64622	0.0096
0.0013	410.0	64780	0.0
0.0166	411.0	64938	0.0285
0.0029	412.0	65096	0.0000
0.0133	413.0	65254	0.0000
0.014	414.0	65412	0.0008
0.0044	415.0	65570	0.0163
0.0077	416.0	65728	0.0000
0.0002	417.0	65886	0.0004
0.0024	418.0	66044	0.0375
0.0092	419.0	66202	0.0008
0.0059	420.0	66360	0.0139
0.0005	421.0	66518	0.0121
0.0128	422.0	66676	0.0105
0.0035	423.0	66834	0.0005
0.0042	424.0	66992	0.0
0.0051	425.0	67150	0.0311
0.0089	426.0	67308	0.0084
0.0112	427.0	67466	0.0229
0.0002	428.0	67624	0.0156
0.0049	429.0	67782	0.0117
0.0051	430.0	67940	0.0150
0.0088	431.0	68098	0.0005
0.01	432.0	68256	0.0136
0.0032	433.0	68414	0.0
0.0003	434.0	68572	0.0155
0.0032	435.0	68730	0.0119
0.0041	436.0	68888	0.0003
0.0	437.0	69046	0.0168
0.0283	438.0	69204	0.0131
0.0032	439.0	69362	0.0000
0.007	440.0	69520	0.0000
0.0079	441.0	69678	0.0251
0.0121	442.0	69836	0.0003
0.0067	443.0	69994	0.0005
0.0017	444.0	70152	0.0000
0.0027	445.0	70310	0.0138
0.0013	446.0	70468	0.0322
0.0086	447.0	70626	0.0000
0.0066	448.0	70784	0.0112
0.003	449.0	70942	0.0000
0.0006	450.0	71100	0.0097
0.01	451.0	71258	0.0113
0.0003	452.0	71416	0.0098
0.0027	453.0	71574	0.0009
0.0034	454.0	71732	0.0119
0.0041	455.0	71890	0.0012
0.01	456.0	72048	0.0222
0.0032	457.0	72206	0.0111
0.0027	458.0	72364	0.0000
0.0064	459.0	72522	0.0149
0.0034	460.0	72680	0.0097
0.0083	461.0	72838	0.0146
0.002	462.0	72996	0.0237
0.004	463.0	73154	0.0135
0.0028	464.0	73312	0.0056
0.0008	465.0	73470	0.0000
0.0015	466.0	73628	0.0004
0.0023	467.0	73786	0.0320
0.008	468.0	73944	0.0108
0.0052	469.0	74102	0.0283
0.0088	470.0	74260	0.0009
0.0037	471.0	74418	0.0251
0.006	472.0	74576	0.0208
0.0008	473.0	74734	0.0237
0.005	474.0	74892	0.0198
0.0033	475.0	75050	0.0000
0.0014	476.0	75208	0.0000
0.0028	477.0	75366	0.0004
0.0082	478.0	75524	0.0149
0.0067	479.0	75682	0.0124
0.0062	480.0	75840	0.0327
0.0035	481.0	75998	0.0273
0.0013	482.0	76156	0.0007
0.0023	483.0	76314	0.0112
0.0042	484.0	76472	0.0101
0.0001	485.0	76630	0.0004
0.0	486.0	76788	0.0000
0.0069	487.0	76946	0.0278
0.0002	488.0	77104	0.0000
0.0045	489.0	77262	0.0163
0.0005	490.0	77420	0.0000
0.0005	491.0	77578	0.0272
0.0014	492.0	77736	0.0000
0.0088	493.0	77894	0.0215
0.0033	494.0	78052	0.0009
0.0031	495.0	78210	0.0261
0.0038	496.0	78368	0.0120
0.0091	497.0	78526	0.0151
0.0062	498.0	78684	0.0009
0.004	499.0	78842	0.0005
0.0025	500.0	79000	0.0134

Framework versions

• Transformers 4.31.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.11.0