Hugging Face's logo

aryaadhi
/
Bloom-Medical-QA-LoRA

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
Bloom-Medical-QA-LoRA / README.md
aryaadhi's picture
aryaadhi
End of training
8514db0 verified
preview code
|
raw
history blame
13.6 kB
metadata
base_model: bigscience/bloom-560m
library_name: peft
license: bigscience-bloom-rail-1.0
tags:
  - generated_from_trainer
model-index:
  - name: Bloom-Medical-QA-LoRA
    results: []

Bloom-Medical-QA-LoRA

This model is a fine-tuned version of bigscience/bloom-560m on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.9682

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • num_epochs: 4

Training results

Training Loss Epoch Step Validation Loss
No log 0.0168 10 2.5135
No log 0.0337 20 2.3298
No log 0.0505 30 2.2756
No log 0.0673 40 2.2419
No log 0.0842 50 2.2270
No log 0.1010 60 2.2102
No log 0.1178 70 2.1931
No log 0.1347 80 2.1846
No log 0.1515 90 2.1749
2.2471 0.1684 100 2.1645
2.2471 0.1852 110 2.1594
2.2471 0.2020 120 2.1557
2.2471 0.2189 130 2.1526
2.2471 0.2357 140 2.1495
2.2471 0.2525 150 2.1498
2.2471 0.2694 160 2.1401
2.2471 0.2862 170 2.1374
2.2471 0.3030 180 2.1317
2.2471 0.3199 190 2.1259
2.1174 0.3367 200 2.1248
2.1174 0.3535 210 2.1240
2.1174 0.3704 220 2.1179
2.1174 0.3872 230 2.1162
2.1174 0.4040 240 2.1163
2.1174 0.4209 250 2.1079
2.1174 0.4377 260 2.1075
2.1174 0.4545 270 2.1035
2.1174 0.4714 280 2.1030
2.1174 0.4882 290 2.0973
2.0744 0.5051 300 2.0895
2.0744 0.5219 310 2.0886
2.0744 0.5387 320 2.0894
2.0744 0.5556 330 2.0908
2.0744 0.5724 340 2.0841
2.0744 0.5892 350 2.0799
2.0744 0.6061 360 2.0799
2.0744 0.6229 370 2.0755
2.0744 0.6397 380 2.0753
2.0744 0.6566 390 2.0713
2.054 0.6734 400 2.0713
2.054 0.6902 410 2.0704
2.054 0.7071 420 2.0678
2.054 0.7239 430 2.0664
2.054 0.7407 440 2.0700
2.054 0.7576 450 2.0672
2.054 0.7744 460 2.0639
2.054 0.7912 470 2.0624
2.054 0.8081 480 2.0630
2.054 0.8249 490 2.0579
2.0228 0.8418 500 2.0548
2.0228 0.8586 510 2.0517
2.0228 0.8754 520 2.0490
2.0228 0.8923 530 2.0509
2.0228 0.9091 540 2.0488
2.0228 0.9259 550 2.0482
2.0228 0.9428 560 2.0446
2.0228 0.9596 570 2.0407
2.0228 0.9764 580 2.0445
2.0228 0.9933 590 2.0410
2.0186 1.0101 600 2.0426
2.0186 1.0269 610 2.0408
2.0186 1.0438 620 2.0407
2.0186 1.0606 630 2.0404
2.0186 1.0774 640 2.0363
2.0186 1.0943 650 2.0358
2.0186 1.1111 660 2.0365
2.0186 1.1279 670 2.0348
2.0186 1.1448 680 2.0343
2.0186 1.1616 690 2.0319
1.9341 1.1785 700 2.0268
1.9341 1.1953 710 2.0287
1.9341 1.2121 720 2.0302
1.9341 1.2290 730 2.0298
1.9341 1.2458 740 2.0307
1.9341 1.2626 750 2.0282
1.9341 1.2795 760 2.0305
1.9341 1.2963 770 2.0240
1.9341 1.3131 780 2.0245
1.9341 1.3300 790 2.0204
1.9309 1.3468 800 2.0208
1.9309 1.3636 810 2.0189
1.9309 1.3805 820 2.0179
1.9309 1.3973 830 2.0185
1.9309 1.4141 840 2.0187
1.9309 1.4310 850 2.0191
1.9309 1.4478 860 2.0218
1.9309 1.4646 870 2.0131
1.9309 1.4815 880 2.0096
1.9309 1.4983 890 2.0092
1.919 1.5152 900 2.0070
1.919 1.5320 910 2.0060
1.919 1.5488 920 2.0070
1.919 1.5657 930 2.0094
1.919 1.5825 940 2.0137
1.919 1.5993 950 2.0092
1.919 1.6162 960 2.0033
1.919 1.6330 970 2.0045
1.919 1.6498 980 2.0053
1.919 1.6667 990 2.0043
1.9546 1.6835 1000 2.0028
1.9546 1.7003 1010 2.0018
1.9546 1.7172 1020 2.0005
1.9546 1.7340 1030 1.9999
1.9546 1.7508 1040 1.9983
1.9546 1.7677 1050 1.9974
1.9546 1.7845 1060 1.9971
1.9546 1.8013 1070 1.9961
1.9546 1.8182 1080 1.9943
1.9546 1.8350 1090 1.9938
1.9247 1.8519 1100 1.9949
1.9247 1.8687 1110 1.9949
1.9247 1.8855 1120 1.9926
1.9247 1.9024 1130 1.9888
1.9247 1.9192 1140 1.9894
1.9247 1.9360 1150 1.9893
1.9247 1.9529 1160 1.9912
1.9247 1.9697 1170 1.9896
1.9247 1.9865 1180 1.9896
1.9247 2.0034 1190 1.9900
1.8978 2.0202 1200 1.9886
1.8978 2.0370 1210 1.9898
1.8978 2.0539 1220 1.9883
1.8978 2.0707 1230 1.9869
1.8978 2.0875 1240 1.9876
1.8978 2.1044 1250 1.9873
1.8978 2.1212 1260 1.9893
1.8978 2.1380 1270 1.9879
1.8978 2.1549 1280 1.9880
1.8978 2.1717 1290 1.9882
1.8637 2.1886 1300 1.9869
1.8637 2.2054 1310 1.9879
1.8637 2.2222 1320 1.9881
1.8637 2.2391 1330 1.9901
1.8637 2.2559 1340 1.9874
1.8637 2.2727 1350 1.9855
1.8637 2.2896 1360 1.9871
1.8637 2.3064 1370 1.9871
1.8637 2.3232 1380 1.9849
1.8637 2.3401 1390 1.9837
1.8206 2.3569 1400 1.9841
1.8206 2.3737 1410 1.9828
1.8206 2.3906 1420 1.9809
1.8206 2.4074 1430 1.9774
1.8206 2.4242 1440 1.9770
1.8206 2.4411 1450 1.9779
1.8206 2.4579 1460 1.9779
1.8206 2.4747 1470 1.9783
1.8206 2.4916 1480 1.9763
1.8206 2.5084 1490 1.9766
1.8587 2.5253 1500 1.9762
1.8587 2.5421 1510 1.9768
1.8587 2.5589 1520 1.9782
1.8587 2.5758 1530 1.9787
1.8587 2.5926 1540 1.9770
1.8587 2.6094 1550 1.9755
1.8587 2.6263 1560 1.9753
1.8587 2.6431 1570 1.9753
1.8587 2.6599 1580 1.9747
1.8587 2.6768 1590 1.9742
1.8226 2.6936 1600 1.9743
1.8226 2.7104 1610 1.9729
1.8226 2.7273 1620 1.9735
1.8226 2.7441 1630 1.9749
1.8226 2.7609 1640 1.9737
1.8226 2.7778 1650 1.9731
1.8226 2.7946 1660 1.9723
1.8226 2.8114 1670 1.9721
1.8226 2.8283 1680 1.9713
1.8226 2.8451 1690 1.9698
1.8331 2.8620 1700 1.9695
1.8331 2.8788 1710 1.9688
1.8331 2.8956 1720 1.9689
1.8331 2.9125 1730 1.9686
1.8331 2.9293 1740 1.9685
1.8331 2.9461 1750 1.9681
1.8331 2.9630 1760 1.9675
1.8331 2.9798 1770 1.9669
1.8331 2.9966 1780 1.9671
1.8331 3.0135 1790 1.9680
1.8251 3.0303 1800 1.9696
1.8251 3.0471 1810 1.9705
1.8251 3.0640 1820 1.9695
1.8251 3.0808 1830 1.9700
1.8251 3.0976 1840 1.9701
1.8251 3.1145 1850 1.9701
1.8251 3.1313 1860 1.9705
1.8251 3.1481 1870 1.9707
1.8251 3.1650 1880 1.9709
1.8251 3.1818 1890 1.9712
1.7809 3.1987 1900 1.9713
1.7809 3.2155 1910 1.9718
1.7809 3.2323 1920 1.9720
1.7809 3.2492 1930 1.9714
1.7809 3.2660 1940 1.9704
1.7809 3.2828 1950 1.9694
1.7809 3.2997 1960 1.9694
1.7809 3.3165 1970 1.9692
1.7809 3.3333 1980 1.9690
1.7809 3.3502 1990 1.9689
1.7957 3.3670 2000 1.9687
1.7957 3.3838 2010 1.9685
1.7957 3.4007 2020 1.9682
1.7957 3.4175 2030 1.9684
1.7957 3.4343 2040 1.9688
1.7957 3.4512 2050 1.9689
1.7957 3.4680 2060 1.9685
1.7957 3.4848 2070 1.9683
1.7957 3.5017 2080 1.9685
1.7957 3.5185 2090 1.9686
1.7859 3.5354 2100 1.9688
1.7859 3.5522 2110 1.9691
1.7859 3.5690 2120 1.9692
1.7859 3.5859 2130 1.9693
1.7859 3.6027 2140 1.9694
1.7859 3.6195 2150 1.9692
1.7859 3.6364 2160 1.9690
1.7859 3.6532 2170 1.9689
1.7859 3.6700 2180 1.9689
1.7859 3.6869 2190 1.9689
1.7794 3.7037 2200 1.9687
1.7794 3.7205 2210 1.9686
1.7794 3.7374 2220 1.9685
1.7794 3.7542 2230 1.9685
1.7794 3.7710 2240 1.9684
1.7794 3.7879 2250 1.9684
1.7794 3.8047 2260 1.9683
1.7794 3.8215 2270 1.9683
1.7794 3.8384 2280 1.9682
1.7794 3.8552 2290 1.9682
1.7766 3.8721 2300 1.9682
1.7766 3.8889 2310 1.9682
1.7766 3.9057 2320 1.9682
1.7766 3.9226 2330 1.9682
1.7766 3.9394 2340 1.9682
1.7766 3.9562 2350 1.9682
1.7766 3.9731 2360 1.9682
1.7766 3.9899 2370 1.9682

Framework versions

  • PEFT 0.11.1
  • Transformers 4.42.4
  • Pytorch 1.13.1+cu117
  • Datasets 2.19.2
  • Tokenizers 0.19.1