Hugging Face's logo

Wenboz
/
llama3-wpo-lora

PEFT
Safetensors
llama
alignment-handbook
trl
dpo
Generated from Trainer
Model card Files Community
Wenboz commited on
Commit
28a42b5
·
verified ·
1 Parent(s): 669c4f7

Model save

Browse files
Files changed (4) hide show
  1. README.md +18 -18
  2. all_results.json +3 -3
  3. train_results.json +3 -3
  4. trainer_state.json +0 -0
README.md CHANGED
@@ -2,9 +2,9 @@
2
  base_model: princeton-nlp/Llama-3-Base-8B-SFT
3
  library_name: peft
4
  tags:
5
- - alignment-handbook
6
  - trl
7
  - dpo
 
8
  - generated_from_trainer
9
  model-index:
10
  - name: llama3-wpo-lora
@@ -18,16 +18,16 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [princeton-nlp/Llama-3-Base-8B-SFT](https://huggingface.co/princeton-nlp/Llama-3-Base-8B-SFT) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.5185
22
- - Rewards/chosen: -0.0112
23
- - Rewards/rejected: -0.8516
24
  - Rewards/accuracies: 0.7280
25
- - Rewards/margins: 0.8404
26
- - Logps/rejected: -285.1924
27
- - Logps/chosen: -292.6592
28
  - Logps/ref Response: -0.5364
29
- - Logits/rejected: -0.3464
30
- - Logits/chosen: -0.3802
31
 
32
  ## Model description
33
 
@@ -64,15 +64,15 @@ The following hyperparameters were used during training:
64
 
65
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logps/ref Response | Logits/rejected | Logits/chosen |
66
  |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:------------------:|:---------------:|:-------------:|
67
- | 0.6207 | 0.1047 | 100 | 0.6043 | 0.2394 | -0.0685 | 0.6860 | 0.3079 | -277.3613 | -290.1534 | -0.5364 | -0.5544 | -0.5590 |
68
- | 0.5646 | 0.2094 | 200 | 0.5509 | -0.0903 | -0.6915 | 0.7220 | 0.6013 | -283.5920 | -293.4499 | -0.5364 | -0.4966 | -0.5149 |
69
- | 0.5411 | 0.3141 | 300 | 0.5362 | -0.1526 | -0.8607 | 0.7280 | 0.7082 | -285.2837 | -294.0728 | -0.5364 | -0.4552 | -0.4787 |
70
- | 0.5103 | 0.4187 | 400 | 0.5296 | -0.1296 | -0.9152 | 0.7160 | 0.7856 | -285.8287 | -293.8433 | -0.5364 | -0.4009 | -0.4304 |
71
- | 0.5408 | 0.5234 | 500 | 0.5238 | -0.0898 | -0.9022 | 0.7280 | 0.8124 | -285.6985 | -293.4456 | -0.5364 | -0.3717 | -0.4037 |
72
- | 0.5216 | 0.6281 | 600 | 0.5218 | 0.0221 | -0.7992 | 0.7380 | 0.8213 | -284.6689 | -292.3262 | -0.5364 | -0.3629 | -0.3958 |
73
- | 0.509 | 0.7328 | 700 | 0.5174 | -0.1835 | -1.0191 | 0.7300 | 0.8356 | -286.8675 | -294.3820 | -0.5364 | -0.3421 | -0.3765 |
74
- | 0.5306 | 0.8375 | 800 | 0.5181 | -0.0066 | -0.8414 | 0.7340 | 0.8348 | -285.0904 | -292.6132 | -0.5364 | -0.3462 | -0.3803 |
75
- | 0.4879 | 0.9422 | 900 | 0.5180 | -0.0159 | -0.8503 | 0.7260 | 0.8343 | -285.1792 | -292.7065 | -0.5364 | -0.3457 | -0.3797 |
76
 
77
 
78
  ### Framework versions
 
2
  base_model: princeton-nlp/Llama-3-Base-8B-SFT
3
  library_name: peft
4
  tags:
 
5
  - trl
6
  - dpo
7
+ - alignment-handbook
8
  - generated_from_trainer
9
  model-index:
10
  - name: llama3-wpo-lora
 
18
 
19
  This model is a fine-tuned version of [princeton-nlp/Llama-3-Base-8B-SFT](https://huggingface.co/princeton-nlp/Llama-3-Base-8B-SFT) on the None dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.5175
22
+ - Rewards/chosen: -0.0517
23
+ - Rewards/rejected: -0.9092
24
  - Rewards/accuracies: 0.7280
25
+ - Rewards/margins: 0.8575
26
+ - Logps/rejected: -285.7691
27
+ - Logps/chosen: -293.0645
28
  - Logps/ref Response: -0.5364
29
+ - Logits/rejected: -0.3072
30
+ - Logits/chosen: -0.3443
31
 
32
  ## Model description
33
 
 
64
 
65
  | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logps/ref Response | Logits/rejected | Logits/chosen |
66
  |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:------------------:|:---------------:|:-------------:|
67
+ | 0.6142 | 0.1047 | 100 | 0.5973 | 0.2024 | -0.1309 | 0.7020 | 0.3333 | -277.9861 | -290.5232 | -0.5364 | -0.5487 | -0.5543 |
68
+ | 0.5579 | 0.2094 | 200 | 0.5483 | -0.0751 | -0.7065 | 0.7120 | 0.6313 | -283.7411 | -293.2985 | -0.5364 | -0.4847 | -0.5042 |
69
+ | 0.5402 | 0.3141 | 300 | 0.5354 | -0.1318 | -0.8578 | 0.7260 | 0.7260 | -285.2545 | -293.8653 | -0.5364 | -0.4387 | -0.4637 |
70
+ | 0.5112 | 0.4187 | 400 | 0.5277 | -0.1698 | -0.9670 | 0.7220 | 0.7973 | -286.3469 | -294.2450 | -0.5364 | -0.3715 | -0.4030 |
71
+ | 0.5319 | 0.5234 | 500 | 0.5212 | -0.1546 | -0.9783 | 0.7260 | 0.8237 | -286.4595 | -294.0932 | -0.5364 | -0.3377 | -0.3727 |
72
+ | 0.5155 | 0.6281 | 600 | 0.5195 | -0.0851 | -0.9285 | 0.7360 | 0.8434 | -285.9612 | -293.3980 | -0.5364 | -0.3247 | -0.3608 |
73
+ | 0.5113 | 0.7328 | 700 | 0.5173 | -0.1941 | -1.0489 | 0.7340 | 0.8547 | -287.1652 | -294.4885 | -0.5364 | -0.3036 | -0.3411 |
74
+ | 0.5268 | 0.8375 | 800 | 0.5177 | -0.0457 | -0.9023 | 0.7220 | 0.8566 | -285.7000 | -293.0044 | -0.5364 | -0.3082 | -0.3453 |
75
+ | 0.4923 | 0.9422 | 900 | 0.5175 | -0.0517 | -0.9092 | 0.7280 | 0.8575 | -285.7691 | -293.0645 | -0.5364 | -0.3072 | -0.3443 |
76
 
77
 
78
  ### Framework versions
all_results.json CHANGED
@@ -15,9 +15,9 @@
15
  "eval_samples_per_second": 5.732,
16
  "eval_steps_per_second": 0.358,
17
  "total_flos": 0.0,
18
- "train_loss": 0.540449628405546,
19
- "train_runtime": 19116.7324,
20
  "train_samples": 61135,
21
- "train_samples_per_second": 3.198,
22
  "train_steps_per_second": 0.05
23
  }
 
15
  "eval_samples_per_second": 5.732,
16
  "eval_steps_per_second": 0.358,
17
  "total_flos": 0.0,
18
+ "train_loss": 0.5383632463934533,
19
+ "train_runtime": 19109.9128,
20
  "train_samples": 61135,
21
+ "train_samples_per_second": 3.199,
22
  "train_steps_per_second": 0.05
23
  }
train_results.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
  "epoch": 0.9997382884061764,
3
  "total_flos": 0.0,
4
- "train_loss": 0.540449628405546,
5
- "train_runtime": 19116.7324,
6
  "train_samples": 61135,
7
- "train_samples_per_second": 3.198,
8
  "train_steps_per_second": 0.05
9
  }
 
1
  {
2
  "epoch": 0.9997382884061764,
3
  "total_flos": 0.0,
4
+ "train_loss": 0.5383632463934533,
5
+ "train_runtime": 19109.9128,
6
  "train_samples": 61135,
7
+ "train_samples_per_second": 3.199,
8
  "train_steps_per_second": 0.05
9
  }
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff