NoteChat-BioBART

This model is a fine-tuned version of checkpoint_global_step_200000 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0020
• Rouge1: 0.0816
• Rouge2: 0.0373
• Rougel: 0.0711
• Rougelsum: 0.074
• Gen Len: 20.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.6113	1.0	3726	3.5345	0.0729	0.0245	0.0639	0.0667	20.0
3.2014	2.0	7452	3.1318	0.0701	0.0228	0.0619	0.0643	20.0
2.9394	3.0	11178	2.8865	0.0733	0.0255	0.0653	0.0678	20.0
2.7238	4.0	14904	2.6827	0.0759	0.0291	0.0673	0.0695	20.0
2.5805	5.0	18630	2.5151	0.0774	0.0311	0.0673	0.07	20.0
2.4169	6.0	22356	2.3876	0.0799	0.0329	0.0686	0.0717	20.0
2.2721	7.0	26082	2.2933	0.081	0.0345	0.0706	0.0734	20.0
2.207	8.0	29808	2.2572	0.0812	0.035	0.071	0.0737	20.0
2.1144	9.0	33534	2.1707	0.081	0.0352	0.0706	0.0735	20.0
2.0559	10.0	37260	2.1287	0.0814	0.0351	0.0694	0.0728	20.0
1.9991	11.0	40986	2.0978	0.081	0.0356	0.0705	0.0734	20.0
1.9552	12.0	44712	2.0716	0.0812	0.0362	0.0709	0.0737	20.0
1.9006	13.0	48438	2.0657	0.081	0.0364	0.0711	0.0739	20.0
1.8592	14.0	52164	2.0483	0.0812	0.0362	0.0704	0.0734	20.0
1.8453	15.0	55890	2.0314	0.0815	0.0375	0.0716	0.0744	20.0
1.8113	16.0	59616	2.0129	0.081	0.0367	0.0708	0.0735	20.0
1.7864	17.0	63342	2.0055	0.0815	0.0371	0.0711	0.074	20.0
1.781	18.0	67068	2.0136	0.0809	0.0368	0.0708	0.0737	20.0
1.7774	19.0	70794	2.0024	0.0815	0.0372	0.071	0.0739	20.0
1.7345	20.0	74520	2.0020	0.0816	0.0373	0.0711	0.074	20.0

Framework versions

• Transformers 4.41.0
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1