Edit model card

Gemma-Medical-QA-LoRA

This model is a fine-tuned version of google/gemma-2b on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.8668

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss
3.2642 0.0103 100 2.9924
2.6906 0.0205 200 2.5289
2.4587 0.0308 300 2.3835
2.357 0.0411 400 2.3593
2.3804 0.0514 500 2.3834
2.3997 0.0616 600 2.4149
2.4127 0.0719 700 2.4326
2.4324 0.0822 800 2.4406
2.4394 0.0924 900 2.4357
2.4363 0.1027 1000 2.4273
2.4104 0.1130 1100 2.4209
2.4255 0.1232 1200 2.4175
2.4019 0.1335 1300 2.4075
2.4133 0.1438 1400 2.4035
2.3963 0.1541 1500 2.3965
2.3874 0.1643 1600 2.3951
2.382 0.1746 1700 2.3907
2.3827 0.1849 1800 2.3838
2.384 0.1951 1900 2.3715
2.3625 0.2054 2000 2.3617
2.353 0.2157 2100 2.3717
2.3554 0.2259 2200 2.3543
2.3426 0.2362 2300 2.3422
2.3522 0.2465 2400 2.3428
2.3355 0.2568 2500 2.3437
2.3451 0.2670 2600 2.3227
2.335 0.2773 2700 2.3303
2.3107 0.2876 2800 2.3312
2.3336 0.2978 2900 2.3223
2.3075 0.3081 3000 2.3116
2.2991 0.3184 3100 2.3155
2.3183 0.3286 3200 2.3094
2.3085 0.3389 3300 2.3035
2.3049 0.3492 3400 2.2998
2.3003 0.3595 3500 2.2944
2.3067 0.3697 3600 2.3003
2.2962 0.3800 3700 2.3012
2.2918 0.3903 3800 2.2859
2.2761 0.4005 3900 2.2826
2.2916 0.4108 4000 2.2703
2.2744 0.4211 4100 2.2754
2.2687 0.4313 4200 2.2646
2.2673 0.4416 4300 2.2654
2.2472 0.4519 4400 2.2617
2.2669 0.4622 4500 2.2536
2.2516 0.4724 4600 2.2585
2.2392 0.4827 4700 2.2514
2.2551 0.4930 4800 2.2445
2.2539 0.5032 4900 2.2380
2.2462 0.5135 5000 2.2374
2.2168 0.5238 5100 2.2376
2.2447 0.5340 5200 2.2272
2.2022 0.5443 5300 2.2081
2.2052 0.5546 5400 2.2144
2.213 0.5649 5500 2.2079
2.2078 0.5751 5600 2.2071
2.199 0.5854 5700 2.2007
2.2037 0.5957 5800 2.1961
2.2061 0.6059 5900 2.1962
2.1766 0.6162 6000 2.1861
2.1852 0.6265 6100 2.1868
2.1859 0.6367 6200 2.1777
2.1891 0.6470 6300 2.1753
2.1611 0.6573 6400 2.1712
2.1444 0.6676 6500 2.1659
2.157 0.6778 6600 2.1505
2.1554 0.6881 6700 2.1399
2.1344 0.6984 6800 2.1402
2.1306 0.7086 6900 2.1252
2.0952 0.7189 7000 2.1077
2.0969 0.7292 7100 2.0913
2.1258 0.7394 7200 2.0886
2.0826 0.7497 7300 2.0780
2.0782 0.7600 7400 2.0791
2.0802 0.7703 7500 2.0685
2.0786 0.7805 7600 2.0647
2.0618 0.7908 7700 2.0588
2.0593 0.8011 7800 2.0619
2.0643 0.8113 7900 2.0614
2.057 0.8216 8000 2.0661
2.0588 0.8319 8100 2.0524
2.0686 0.8421 8200 2.0546
2.0333 0.8524 8300 2.0496
2.0621 0.8627 8400 2.0474
2.023 0.8730 8500 2.0390
2.0507 0.8832 8600 2.0414
2.0476 0.8935 8700 2.0421
2.033 0.9038 8800 2.0400
2.0617 0.9140 8900 2.0383
2.0287 0.9243 9000 2.0441
2.0156 0.9346 9100 2.0335
2.0207 0.9448 9200 2.0337
2.036 0.9551 9300 2.0236
2.0411 0.9654 9400 2.0360
2.0277 0.9757 9500 2.0262
2.0449 0.9859 9600 2.0298
2.0323 0.9962 9700 2.0287
2.0124 1.0065 9800 2.0220
2.0211 1.0167 9900 2.0266
2.0137 1.0270 10000 2.0147
2.0169 1.0373 10100 2.0132
2.0075 1.0476 10200 2.0189
2.0022 1.0578 10300 2.0218
2.0172 1.0681 10400 2.0088
2.0152 1.0784 10500 2.0121
1.9993 1.0886 10600 2.0136
2.0109 1.0989 10700 2.0035
2.0155 1.1092 10800 2.0092
2.0101 1.1194 10900 2.0123
2.0028 1.1297 11000 2.0066
2.005 1.1400 11100 2.0060
2.0002 1.1503 11200 2.0026
1.994 1.1605 11300 2.0017
1.9938 1.1708 11400 2.0023
1.9896 1.1811 11500 2.0062
1.9957 1.1913 11600 1.9995
1.987 1.2016 11700 1.9973
2.0086 1.2119 11800 1.9899
1.994 1.2221 11900 1.9962
1.9668 1.2324 12000 1.9951
2.0094 1.2427 12100 1.9944
1.9898 1.2530 12200 1.9907
1.9958 1.2632 12300 1.9728
2.0 1.2735 12400 1.9864
1.9818 1.2838 12500 1.9907
2.0054 1.2940 12600 1.9877
1.9837 1.3043 12700 1.9925
1.9938 1.3146 12800 1.9869
2.0015 1.3248 12900 1.9958
1.9753 1.3351 13000 1.9873
1.9632 1.3454 13100 1.9920
1.9932 1.3557 13200 1.9782
1.9875 1.3659 13300 1.9777
1.9693 1.3762 13400 1.9826
1.9898 1.3865 13500 1.9748
1.9678 1.3967 13600 1.9892
1.989 1.4070 13700 1.9770
1.9789 1.4173 13800 1.9684
1.9733 1.4275 13900 1.9726
1.9623 1.4378 14000 1.9726
1.95 1.4481 14100 1.9733
1.9679 1.4584 14200 1.9689
1.964 1.4686 14300 1.9788
2.0041 1.4789 14400 1.9707
1.9626 1.4892 14500 1.9641
1.979 1.4994 14600 1.9682
1.9558 1.5097 14700 1.9677
1.9701 1.5200 14800 1.9677
1.976 1.5302 14900 1.9659
1.9448 1.5405 15000 1.9627
1.9494 1.5508 15100 1.9698
1.9552 1.5611 15200 1.9746
1.9628 1.5713 15300 1.9635
1.9589 1.5816 15400 1.9668
1.9769 1.5919 15500 1.9625
1.9513 1.6021 15600 1.9599
1.9578 1.6124 15700 1.9638
1.9694 1.6227 15800 1.9612
1.9508 1.6329 15900 1.9642
1.9568 1.6432 16000 1.9583
1.9474 1.6535 16100 1.9610
1.9616 1.6638 16200 1.9591
1.963 1.6740 16300 1.9621
1.9612 1.6843 16400 1.9602
1.959 1.6946 16500 1.9417
1.945 1.7048 16600 1.9535
1.9519 1.7151 16700 1.9553
1.9361 1.7254 16800 1.9538
1.9411 1.7356 16900 1.9450
1.9614 1.7459 17000 1.9579
1.9634 1.7562 17100 1.9546
1.9559 1.7665 17200 1.9501
1.9483 1.7767 17300 1.9491
1.9603 1.7870 17400 1.9529
1.9559 1.7973 17500 1.9495
1.9464 1.8075 17600 1.9491
1.9598 1.8178 17700 1.9499
1.9349 1.8281 17800 1.9512
1.9515 1.8383 17900 1.9454
1.9481 1.8486 18000 1.9423
1.9479 1.8589 18100 1.9486
1.944 1.8692 18200 1.9579
1.9333 1.8794 18300 1.9406
1.9489 1.8897 18400 1.9439
1.9603 1.9000 18500 1.9437
1.9349 1.9102 18600 1.9366
1.9453 1.9205 18700 1.9440
1.9199 1.9308 18800 1.9397
1.9554 1.9410 18900 1.9443
1.9256 1.9513 19000 1.9345
1.9393 1.9616 19100 1.9372
1.9432 1.9719 19200 1.9392
1.935 1.9821 19300 1.9379
1.9252 1.9924 19400 1.9459
1.9378 2.0027 19500 1.9403
1.9326 2.0129 19600 1.9400
1.9272 2.0232 19700 1.9380
1.9368 2.0335 19800 1.9408
1.9318 2.0438 19900 1.9284
1.9165 2.0540 20000 1.9395
1.941 2.0643 20100 1.9335
1.9236 2.0746 20200 1.9384
1.9222 2.0848 20300 1.9318
1.9242 2.0951 20400 1.9293
1.9262 2.1054 20500 1.9298
1.9149 2.1156 20600 1.9291
1.9226 2.1259 20700 1.9338
1.9393 2.1362 20800 1.9306
1.9294 2.1465 20900 1.9358
1.9275 2.1567 21000 1.9345
1.918 2.1670 21100 1.9332
1.9164 2.1773 21200 1.9240
1.9248 2.1875 21300 1.9302
1.9128 2.1978 21400 1.9341
1.9159 2.2081 21500 1.9217
1.934 2.2183 21600 1.9261
1.9258 2.2286 21700 1.9244
1.9034 2.2389 21800 1.9384
1.9023 2.2492 21900 1.9233
1.94 2.2594 22000 1.9232
1.9319 2.2697 22100 1.9289
1.8986 2.2800 22200 1.9191
1.9087 2.2902 22300 1.9196
1.9125 2.3005 22400 1.9235
1.9081 2.3108 22500 1.9103
1.9324 2.3210 22600 1.9249
1.9132 2.3313 22700 1.9210
1.9134 2.3416 22800 1.9273
1.923 2.3519 22900 1.9184
1.9271 2.3621 23000 1.9240
1.9089 2.3724 23100 1.9195
1.9232 2.3827 23200 1.9158
1.9153 2.3929 23300 1.9159
1.9303 2.4032 23400 1.9214
1.9267 2.4135 23500 1.9195
1.9041 2.4237 23600 1.9159
1.9104 2.4340 23700 1.9119
1.9126 2.4443 23800 1.9147
1.9179 2.4546 23900 1.9173
1.8934 2.4648 24000 1.9156
1.9213 2.4751 24100 1.9168
1.9244 2.4854 24200 1.9193
1.9064 2.4956 24300 1.9014
1.8898 2.5059 24400 1.9215
1.9063 2.5162 24500 1.9200
1.9105 2.5264 24600 1.9210
1.916 2.5367 24700 1.9113
1.9099 2.5470 24800 1.9090
1.9051 2.5573 24900 1.9060
1.9177 2.5675 25000 1.9181
1.923 2.5778 25100 1.9144
1.8971 2.5881 25200 1.9079
1.9133 2.5983 25300 1.9068
1.9318 2.6086 25400 1.9089
1.9149 2.6189 25500 1.9109
1.9145 2.6291 25600 1.9076
1.911 2.6394 25700 1.9149
1.8884 2.6497 25800 1.9018
1.8946 2.6600 25900 1.9217
1.9106 2.6702 26000 1.9082
1.906 2.6805 26100 1.9063
1.9026 2.6908 26200 1.9070
1.9088 2.7010 26300 1.9059
1.8938 2.7113 26400 1.9019
1.8964 2.7216 26500 1.9131
1.8947 2.7318 26600 1.9096
1.8906 2.7421 26700 1.9053
1.911 2.7524 26800 1.8969
1.887 2.7627 26900 1.9111
1.8864 2.7729 27000 1.9064
1.9195 2.7832 27100 1.9044
1.9129 2.7935 27200 1.9019
1.8915 2.8037 27300 1.8970
1.9035 2.8140 27400 1.9049
1.8755 2.8243 27500 1.8986
1.8939 2.8345 27600 1.8993
1.895 2.8448 27700 1.8978
1.8744 2.8551 27800 1.9052
1.9178 2.8654 27900 1.9040
1.8916 2.8756 28000 1.8977
1.9226 2.8859 28100 1.9014
1.8772 2.8962 28200 1.8990
1.9011 2.9064 28300 1.8888
1.8891 2.9167 28400 1.8998
1.91 2.9270 28500 1.8976
1.9288 2.9372 28600 1.8976
1.8759 2.9475 28700 1.9000
1.8806 2.9578 28800 1.9029
1.8971 2.9681 28900 1.8981
1.9036 2.9783 29000 1.8944
1.8898 2.9886 29100 1.8983
1.8935 2.9989 29200 1.8930
1.8838 3.0091 29300 1.9015
1.8964 3.0194 29400 1.9020
1.895 3.0297 29500 1.8909
1.8864 3.0400 29600 1.8871
1.89 3.0502 29700 1.8857
1.8846 3.0605 29800 1.8958
1.8983 3.0708 29900 1.8910
1.8917 3.0810 30000 1.8936
1.8913 3.0913 30100 1.8986
1.8559 3.1016 30200 1.9011
1.9041 3.1118 30300 1.8858
1.8752 3.1221 30400 1.8983
1.8813 3.1324 30500 1.8963
1.877 3.1427 30600 1.8924
1.8724 3.1529 30700 1.8895
1.8946 3.1632 30800 1.8942
1.8872 3.1735 30900 1.8940
1.8847 3.1837 31000 1.8931
1.8729 3.1940 31100 1.8850
1.8724 3.2043 31200 1.8911
1.864 3.2145 31300 1.8923
1.8824 3.2248 31400 1.8925
1.9007 3.2351 31500 1.8945
1.8993 3.2454 31600 1.8867
1.8687 3.2556 31700 1.8937
1.8768 3.2659 31800 1.8842
1.8744 3.2762 31900 1.8933
1.8877 3.2864 32000 1.8901
1.8644 3.2967 32100 1.8798
1.8631 3.3070 32200 1.8890
1.8833 3.3172 32300 1.8862
1.8889 3.3275 32400 1.8835
1.8934 3.3378 32500 1.8858
1.8905 3.3481 32600 1.8856
1.8965 3.3583 32700 1.8882
1.8833 3.3686 32800 1.8845
1.9017 3.3789 32900 1.8835
1.8885 3.3891 33000 1.8821
1.8852 3.3994 33100 1.8903
1.8727 3.4097 33200 1.8796
1.8788 3.4199 33300 1.8892
1.8609 3.4302 33400 1.8792
1.9 3.4405 33500 1.8748
1.8731 3.4508 33600 1.8825
1.8753 3.4610 33700 1.8773
1.8651 3.4713 33800 1.8853
1.8757 3.4816 33900 1.8819
1.8939 3.4918 34000 1.8832
1.8939 3.5021 34100 1.8811
1.8719 3.5124 34200 1.8812
1.859 3.5226 34300 1.8834
1.8866 3.5329 34400 1.8762
1.888 3.5432 34500 1.8834
1.8816 3.5535 34600 1.8836
1.8855 3.5637 34700 1.8837
1.8731 3.5740 34800 1.8829
1.9 3.5843 34900 1.8847
1.8767 3.5945 35000 1.8773
1.8847 3.6048 35100 1.8841
1.8716 3.6151 35200 1.8773
1.8746 3.6253 35300 1.8905
1.8672 3.6356 35400 1.8835
1.8825 3.6459 35500 1.8796
1.8711 3.6562 35600 1.8781
1.873 3.6664 35700 1.8787
1.8841 3.6767 35800 1.8774
1.8668 3.6870 35900 1.8827
1.8642 3.6972 36000 1.8804
1.8813 3.7075 36100 1.8761
1.8602 3.7178 36200 1.8772
1.8772 3.7280 36300 1.8859
1.8847 3.7383 36400 1.8792
1.8737 3.7486 36500 1.8823
1.8683 3.7589 36600 1.8812
1.8731 3.7691 36700 1.8808
1.8467 3.7794 36800 1.8828
1.8877 3.7897 36900 1.8820
1.8751 3.7999 37000 1.8840
1.8967 3.8102 37100 1.8824
1.8898 3.8205 37200 1.8747
1.8772 3.8307 37300 1.8736
1.8989 3.8410 37400 1.8733
1.8636 3.8513 37500 1.8791
1.8806 3.8616 37600 1.8731
1.8825 3.8718 37700 1.8811
1.8693 3.8821 37800 1.8753
1.8534 3.8924 37900 1.8731
1.867 3.9026 38000 1.8765
1.876 3.9129 38100 1.8801
1.848 3.9232 38200 1.8709
1.8628 3.9334 38300 1.8739
1.8634 3.9437 38400 1.8734
1.8691 3.9540 38500 1.8747
1.8676 3.9643 38600 1.8748
1.874 3.9745 38700 1.8745
1.8757 3.9848 38800 1.8797
1.8573 3.9951 38900 1.8741
1.8578 4.0053 39000 1.8765
1.8755 4.0156 39100 1.8761
1.885 4.0259 39200 1.8760
1.8647 4.0362 39300 1.8759
1.8642 4.0464 39400 1.8760
1.8838 4.0567 39500 1.8747
1.8632 4.0670 39600 1.8747
1.881 4.0772 39700 1.8758
1.8661 4.0875 39800 1.8699
1.8748 4.0978 39900 1.8723
1.8593 4.1080 40000 1.8688
1.8781 4.1183 40100 1.8698
1.847 4.1286 40200 1.8751
1.8534 4.1389 40300 1.8682
1.856 4.1491 40400 1.8704
1.8735 4.1594 40500 1.8710
1.8586 4.1697 40600 1.8695
1.8466 4.1799 40700 1.8686
1.8594 4.1902 40800 1.8692
1.86 4.2005 40900 1.8674
1.8643 4.2107 41000 1.8693
1.8446 4.2210 41100 1.8685
1.8578 4.2313 41200 1.8710
1.8473 4.2416 41300 1.8716
1.865 4.2518 41400 1.8724
1.848 4.2621 41500 1.8731
1.864 4.2724 41600 1.8684
1.8584 4.2826 41700 1.8753
1.842 4.2929 41800 1.8666
1.8735 4.3032 41900 1.8692
1.8621 4.3134 42000 1.8674
1.8604 4.3237 42100 1.8626
1.8586 4.3340 42200 1.8696
1.8914 4.3443 42300 1.8684
1.8752 4.3545 42400 1.8671
1.8856 4.3648 42500 1.8719
1.8712 4.3751 42600 1.8729
1.8535 4.3853 42700 1.8719
1.8787 4.3956 42800 1.8674
1.8659 4.4059 42900 1.8709
1.8818 4.4161 43000 1.8713
1.8459 4.4264 43100 1.8723
1.8766 4.4367 43200 1.8668
1.8629 4.4470 43300 1.8678
1.8594 4.4572 43400 1.8687
1.8589 4.4675 43500 1.8716
1.8755 4.4778 43600 1.8665
1.8526 4.4880 43700 1.8656
1.8675 4.4983 43800 1.8727
1.8503 4.5086 43900 1.8705
1.8606 4.5188 44000 1.8736
1.8677 4.5291 44100 1.8683
1.8571 4.5394 44200 1.8712
1.8752 4.5497 44300 1.8710
1.8528 4.5599 44400 1.8697
1.8815 4.5702 44500 1.8689
1.8768 4.5805 44600 1.8672
1.8815 4.5907 44700 1.8740
1.8539 4.6010 44800 1.8704
1.8776 4.6113 44900 1.8652
1.8446 4.6215 45000 1.8678
1.8704 4.6318 45100 1.8683
1.8522 4.6421 45200 1.8668
1.8827 4.6524 45300 1.8673
1.851 4.6626 45400 1.8652
1.8577 4.6729 45500 1.8638
1.8581 4.6832 45600 1.8694
1.851 4.6934 45700 1.8675
1.8563 4.7037 45800 1.8704
1.8778 4.7140 45900 1.8656
1.8597 4.7242 46000 1.8676
1.8501 4.7345 46100 1.8684
1.8608 4.7448 46200 1.8684
1.8609 4.7551 46300 1.8687
1.8681 4.7653 46400 1.8649
1.8625 4.7756 46500 1.8672
1.8467 4.7859 46600 1.8681
1.8359 4.7961 46700 1.8669
1.855 4.8064 46800 1.8667
1.8551 4.8167 46900 1.8662
1.8658 4.8269 47000 1.8655
1.8621 4.8372 47100 1.8682
1.8679 4.8475 47200 1.8691
1.8653 4.8578 47300 1.8669
1.8427 4.8680 47400 1.8645
1.8538 4.8783 47500 1.8658
1.8617 4.8886 47600 1.8656
1.8693 4.8988 47700 1.8661
1.8379 4.9091 47800 1.8668
1.8709 4.9194 47900 1.8673
1.8537 4.9296 48000 1.8664
1.8568 4.9399 48100 1.8664
1.867 4.9502 48200 1.8661
1.8591 4.9605 48300 1.8658
1.8565 4.9707 48400 1.8661
1.8541 4.9810 48500 1.8667
1.8736 4.9913 48600 1.8668

Framework versions

  • PEFT 0.12.0
  • Transformers 4.43.3
  • Pytorch 1.13.1+cu117
  • Datasets 2.19.2
  • Tokenizers 0.19.1
Downloads last month
6
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for aryaadhi/Gemma-Medical-QA-LoRA

Base model

google/gemma-2b
Adapter
(23002)
this model