fine-tuned-2048-inputs-30-epochs

This model is a fine-tuned version of GanjinZero/biobart-v2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8273
  • Rouge1: 0.2909
  • Rouge2: 0.117
  • Rougel: 0.2667
  • Rougelsum: 0.2666
  • Gen Len: 15.53

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 151 0.7529 0.2073 0.0784 0.1893 0.1893 13.16
No log 2.0 302 0.7144 0.2569 0.0828 0.2249 0.2262 13.76
No log 3.0 453 0.6993 0.2397 0.0787 0.2138 0.2143 14.52
0.7226 4.0 604 0.6957 0.2857 0.1014 0.2603 0.2619 14.27
0.7226 5.0 755 0.7037 0.2906 0.1168 0.2653 0.268 14.73
0.7226 6.0 906 0.6971 0.2753 0.1126 0.2512 0.2512 14.92
0.4948 7.0 1057 0.7117 0.2806 0.1139 0.2546 0.2539 14.93
0.4948 8.0 1208 0.7185 0.2931 0.1188 0.2667 0.2685 14.45
0.4948 9.0 1359 0.7250 0.3007 0.1106 0.2736 0.2755 14.76
0.368 10.0 1510 0.7343 0.3157 0.126 0.2908 0.2904 14.67
0.368 11.0 1661 0.7418 0.3045 0.1194 0.2758 0.2757 15.23
0.368 12.0 1812 0.7521 0.2981 0.113 0.2745 0.275 14.91
0.368 13.0 1963 0.7556 0.2902 0.1142 0.2695 0.2712 15.01
0.2865 14.0 2114 0.7636 0.3145 0.1238 0.2936 0.295 15.44
0.2865 15.0 2265 0.7722 0.2965 0.1102 0.2684 0.2704 14.93
0.2865 16.0 2416 0.7788 0.3015 0.1087 0.2737 0.2751 15.29
0.2221 17.0 2567 0.7834 0.2957 0.1127 0.2691 0.2688 15.11
0.2221 18.0 2718 0.7905 0.292 0.1136 0.2595 0.2596 15.1
0.2221 19.0 2869 0.7945 0.2903 0.1027 0.2626 0.263 15.5
0.1825 20.0 3020 0.8033 0.3146 0.1226 0.2826 0.2839 15.54
0.1825 21.0 3171 0.8009 0.3027 0.114 0.2742 0.2749 15.69
0.1825 22.0 3322 0.8085 0.2951 0.1132 0.2616 0.2624 15.37
0.1825 23.0 3473 0.8120 0.3045 0.1182 0.2733 0.2749 15.48
0.1498 24.0 3624 0.8163 0.3015 0.111 0.2723 0.2738 15.47
0.1498 25.0 3775 0.8197 0.3054 0.1144 0.2785 0.2778 15.51
0.1498 26.0 3926 0.8212 0.2987 0.1199 0.2723 0.2709 15.59
0.1329 27.0 4077 0.8230 0.3025 0.1154 0.2751 0.2756 15.5
0.1329 28.0 4228 0.8250 0.2845 0.1108 0.2599 0.2608 15.49
0.1329 29.0 4379 0.8275 0.3002 0.1102 0.2745 0.2753 15.57
0.1226 30.0 4530 0.8273 0.2909 0.117 0.2667 0.2666 15.53

Framework versions

  • Transformers 4.36.2
  • Pytorch 1.12.1+cu113
  • Datasets 2.15.0
  • Tokenizers 0.15.0
Downloads last month
16
Safetensors
Model size
166M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for tanatapanun/fine-tuned-2048-inputs-30-epochs

Finetuned
(13)
this model