Model save

README.md

@@ -0,0 +1,76 @@
+---
+license: other
+library_name: peft
+tags:
+- trl
+- dpo
+- generated_from_trainer
+base_model: meta-llama/Meta-Llama-3-8B
+model-index:
+- name: llama3-L1-SFT-L2-DPO
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# llama3-L1-SFT-L2-DPO
+
+This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0028
+- Rewards/chosen: -0.4016
+- Rewards/rejected: -9.3312
+- Rewards/accuracies: 1.0
+- Rewards/margins: 8.9296
+- Logps/rejected: -994.2072
+- Logps/chosen: -84.5141
+- Logits/rejected: 1.5698
+- Logits/chosen: 0.6541
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 4
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.0033        | 0.2497 | 1000 | 0.0065          | -0.3610        | -7.6953          | 1.0                | 7.3344          | -830.6234      | -80.4518     | 1.5489          | 0.7452        |
+| 0.0013        | 0.4995 | 2000 | 0.0031          | -0.3798        | -9.1892          | 1.0                | 8.8094          | -980.0131      | -82.3365     | 1.5546          | 0.6455        |
+| 0.0019        | 0.7492 | 3000 | 0.0028          | -0.3966        | -9.3440          | 1.0                | 8.9474          | -995.4902      | -84.0208     | 1.5703          | 0.6568        |
+| 0.0011        | 0.9989 | 4000 | 0.0028          | -0.4016        | -9.3312          | 1.0                | 8.9296          | -994.2072      | -84.5141     | 1.5698          | 0.6541        |
+
+
+### Framework versions
+
+- PEFT 0.7.1
+- Transformers 4.40.2
+- Pytorch 2.1.2+cu121
+- Datasets 2.18.0
+- Tokenizers 0.19.1

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b607da40eb5ad6af6d94bf7e1407aec44f32e473d80b2a1409d0ce1d04fc52d8
+oid sha256:50cb2502b1c876c5aa351822c0eecad19176456eb3fe2244480ef0346afd007c
 size 83946192

all_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 0.9999375663357682,
+    "total_flos": 0.0,
+    "train_loss": 0.05122942747121405,
+    "train_runtime": 6577.5594,
+    "train_samples": 16017,
+    "train_samples_per_second": 2.435,
+    "train_steps_per_second": 0.609
+}

runs/May10_08-42-30_612e66badb5c/events.out.tfevents.1715330677.612e66badb5c.57653.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5a53e4b65765cf445079c4bb9edc660cfdf22d4e81078fa7ec16f59f518a6dd5
-size 283610
+oid sha256:d6e8d9d7e668d238b2f68fe40c66ab16531c25936ae0ca49e2c63c37a66922dc
+size 283964

train_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 0.9999375663357682,
+    "total_flos": 0.0,
+    "train_loss": 0.05122942747121405,
+    "train_runtime": 6577.5594,
+    "train_samples": 16017,
+    "train_samples_per_second": 2.435,
+    "train_steps_per_second": 0.609
+}