llama3.1_8b_dpo_bwgenerator

README.md

@@ -17,6 +17,16 @@ should probably proofread and complete it, then remove this comment. -->
 # llama3.1_8b_dpo_bwgenerator
 
 This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0426
+- Rewards/chosen: -9.1334
+- Rewards/rejected: -41.2562
+- Rewards/accuracies: 0.9968
+- Rewards/margins: 32.1228
+- Logps/rejected: -519.8535
+- Logps/chosen: -173.9601
+- Logits/rejected: -1.2773
+- Logits/chosen: -1.8503
 
 ## Model description
 
@@ -37,7 +47,7 @@ More information needed
 The following hyperparameters were used during training:
 - learning_rate: 5e-05
 - train_batch_size: 4
-- eval_batch_size: 8
+- eval_batch_size: 4
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
@@ -45,6 +55,21 @@ The following hyperparameters were used during training:
 
 ### Training results
 
+| Training Loss | Epoch  | Step  | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:------:|:-----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.0892        | 0.0719 | 1000  | 0.1265          | -30.3353       | -68.4770         | 0.9900             | 38.1417         | -792.0620      | -385.9789    | -1.2593         | -1.8522       |
+| 0.0747        | 0.1438 | 2000  | 0.0675          | -15.0547       | -42.3699         | 0.9932             | 27.3152         | -530.9907      | -233.1734    | -1.2465         | -1.8204       |
+| 0.0508        | 0.2157 | 3000  | 0.0497          | -10.9548       | -36.2994         | 0.9951             | 25.3446         | -470.2862      | -192.1746    | -1.2443         | -1.8188       |
+| 0.0342        | 0.2876 | 4000  | 0.0521          | -11.1133       | -39.6413         | 0.9945             | 28.5280         | -503.7046      | -193.7589    | -1.2760         | -1.8503       |
+| 0.0413        | 0.3595 | 5000  | 0.0489          | -10.1717       | -41.3975         | 0.9958             | 31.2258         | -521.2673      | -184.3437    | -1.2904         | -1.8675       |
+| 0.0359        | 0.4313 | 6000  | 0.0465          | -10.0814       | -40.8284         | 0.9961             | 30.7471         | -515.5765      | -183.4398    | -1.2872         | -1.8619       |
+| 0.0431        | 0.5032 | 7000  | 0.0509          | -12.0232       | -46.5198         | 0.9961             | 34.4966         | -572.4903      | -202.8584    | -1.2039         | -1.7967       |
+| 0.0549        | 0.5751 | 8000  | 0.0440          | -9.5094        | -40.3277         | 0.9958             | 30.8183         | -510.5694      | -177.7208    | -1.2813         | -1.8498       |
+| 0.0424        | 0.6470 | 9000  | 0.0440          | -9.7038        | -41.6696         | 0.9964             | 31.9658         | -523.9880      | -179.6640    | -1.2597         | -1.8373       |
+| 0.0231        | 0.7189 | 10000 | 0.0450          | -9.8079        | -43.3146         | 0.9964             | 33.5067         | -540.4382      | -180.7056    | -1.2646         | -1.8409       |
+| 0.0225        | 0.7908 | 11000 | 0.0436          | -9.1686        | -41.6580         | 0.9968             | 32.4894         | -523.8720      | -174.3121    | -1.2758         | -1.8486       |
+| 0.0309        | 0.8627 | 12000 | 0.0432          | -9.4044        | -41.9421         | 0.9968             | 32.5377         | -526.7131      | -176.6707    | -1.2731         | -1.8472       |
+| 0.0567        | 0.9346 | 13000 | 0.0426          | -9.1334        | -41.2562         | 0.9968             | 32.1228         | -519.8535      | -173.9601    | -1.2773         | -1.8503       |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6013fb6cbd103444b967e1b496fb4bffa8f8ccf08a7b1452d992d24c1f87dd9e
+oid sha256:b88ca0cc6903f09fd2fd040177567dd3a979a6a86b8120561138f3b480b2a7bd
 size 6832728

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:00208183eecae9d1a7cbd86499e6530c1bd1295cc3b04a6e3cac6cdf0803f99b
-size 6072
+oid sha256:31df9dfb872337e9eed8594b960a9549b0b9e038ace062b07f394d56d573d7ad
+size 6008