GerMedBert_ATTR_V02_BRONCO

This model is a fine-tuned version of GerMedBERT/medbert-512 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0593
• F1 Score: 0.8187
• Precision: 0.8235
• Recall: 0.8140
• Accuracy: 0.8993
• Num Input Tokens Seen: 4204246

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 6

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy	Input Tokens Seen
No log	0.25	81	0.1949	0.0	1.0	0.0	0.7240	175856
0.206	0.5	162	0.1196	0.4123	0.8393	0.2733	0.7847	349280
0.206	0.75	243	0.0945	0.6296	0.8673	0.4942	0.8351	526096
0.1019	1.0	324	0.0792	0.7584	0.8	0.7209	0.8698	702407
0.1019	1.25	405	0.0697	0.7874	0.7784	0.7965	0.8785	877047
0.0715	1.5	486	0.0697	0.7778	0.7447	0.8140	0.8715	1052647
0.0715	1.75	567	0.0673	0.7568	0.7826	0.7326	0.8715	1227287
0.0638	2.0	648	0.0680	0.7781	0.7358	0.8256	0.8715	1400330
0.0638	2.25	729	0.0622	0.7965	0.8084	0.7849	0.8941	1577786
0.0445	2.5	810	0.0593	0.8012	0.7943	0.8081	0.8906	1751722
0.0445	2.75	891	0.0583	0.8023	0.8023	0.8023	0.8906	1927258
0.0396	3.0	972	0.0579	0.8161	0.8068	0.8256	0.8993	2100749
0.0396	3.25	1053	0.0598	0.8125	0.7944	0.8314	0.8941	2276989
0.0289	3.5	1134	0.0592	0.8036	0.8232	0.7849	0.8941	2451501
0.0289	3.75	1215	0.0585	0.7954	0.7886	0.8023	0.8906	2628573
0.0271	4.0	1296	0.0571	0.8171	0.8034	0.8314	0.8993	2802576
0.0271	4.25	1377	0.0581	0.8235	0.8333	0.8140	0.9045	2978368
0.0194	4.5	1458	0.0619	0.7978	0.7717	0.8256	0.8837	3154544
0.0194	4.75	1539	0.0612	0.8048	0.8323	0.7791	0.8958	3330400
0.0193	5.0	1620	0.0585	0.8059	0.8155	0.7965	0.8958	3505555
0.0193	5.25	1701	0.0587	0.8187	0.8235	0.8140	0.9010	3680771
0.0153	5.5	1782	0.0592	0.8242	0.8171	0.8314	0.9010	3856947
0.0153	5.75	1863	0.0592	0.8163	0.8187	0.8140	0.8993	4030371
0.0146	6.0	1944	0.0593	0.8187	0.8235	0.8140	0.8993	4204246

Framework versions

• Transformers 4.40.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1