Hugging Face's logo

nedith22
/
lora_fine_tuned_bart

PEFT
Safetensors
Generated from Trainer
Model card Files Community
lora_fine_tuned_bart / README.md
nedith22's picture
nedith22
End of training
bd074a1 verified
preview code
|
raw
history blame contribute delete
4.07 kB
metadata
base_model: facebook/bart-large-cnn
library_name: peft
license: mit
tags:
  - generated_from_trainer
model-index:
  - name: lora_fine_tuned_bart
    results: []

Visualize in Weights & Biases

lora_fine_tuned_bart

This model is a fine-tuned version of facebook/bart-large-cnn on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6906

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 4e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 50

Training results

Training Loss Epoch Step Validation Loss
1.7253 1.0 32 1.8719
1.426 2.0 64 1.6167
1.3662 3.0 96 1.5225
1.3049 4.0 128 1.4695
1.2084 5.0 160 1.4319
1.2666 6.0 192 1.3709
1.2344 7.0 224 1.3053
1.1056 8.0 256 1.2560
1.025 9.0 288 1.1773
0.915 10.0 320 1.0743
0.8726 11.0 352 1.0085
0.8281 12.0 384 0.9630
0.777 13.0 416 0.9116
0.7681 14.0 448 0.8817
0.664 15.0 480 0.8357
0.6604 16.0 512 0.8077
0.6351 17.0 544 0.7837
0.6455 18.0 576 0.7724
0.6167 19.0 608 0.7585
0.5969 20.0 640 0.7443
0.5605 21.0 672 0.7382
0.5835 22.0 704 0.7302
0.5668 23.0 736 0.7183
0.575 24.0 768 0.7124
0.5319 25.0 800 0.7129
0.5515 26.0 832 0.7085
0.5219 27.0 864 0.7119
0.5509 28.0 896 0.7074
0.5172 29.0 928 0.7014
0.5298 30.0 960 0.7034
0.5071 31.0 992 0.6930
0.525 32.0 1024 0.6941
0.5153 33.0 1056 0.6963
0.5115 34.0 1088 0.6925
0.5194 35.0 1120 0.6933
0.5138 36.0 1152 0.6926
0.4649 37.0 1184 0.6913
0.5127 38.0 1216 0.6932
0.5044 39.0 1248 0.6929
0.4701 40.0 1280 0.6921
0.5156 41.0 1312 0.6931
0.5163 42.0 1344 0.6898
0.5153 43.0 1376 0.6896
0.5054 44.0 1408 0.6880
0.4915 45.0 1440 0.6872
0.4908 46.0 1472 0.6879
0.4836 47.0 1504 0.6891
0.491 48.0 1536 0.6889
0.4814 49.0 1568 0.6905
0.4872 50.0 1600 0.6906

Framework versions

  • PEFT 0.12.0
  • Transformers 4.42.3
  • Pytorch 2.1.2
  • Datasets 2.20.0
  • Tokenizers 0.19.1