Hugging Face's logo

nlile
/
zephyr-7b-dpo-full

Text Generation
Transformers
TensorBoard
Safetensors
mistral
Generated from Trainer
conversational
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
nlile commited on
Commit
0da90ed
·
1 Parent(s): 9c7ea99

Model save

Browse files
Files changed (18) hide show
  1. README.md +134 -0
  2. all_results.json +21 -0
  3. config.json +25 -0
  4. eval_results.json +16 -0
  5. generation_config.json +6 -0
  6. model-00001-of-00003.safetensors +3 -0
  7. model-00002-of-00003.safetensors +3 -0
  8. model-00003-of-00003.safetensors +3 -0
  9. model.safetensors.index.json +298 -0
  10. runs/Nov27_20-27-33_209-20-157-49/events.out.tfevents.1701116942.209-20-157-49.939604.0 +3 -0
  11. runs/Nov27_20-35-12_209-20-157-49/events.out.tfevents.1701117420.209-20-157-49.941709.0 +3 -0
  12. runs/Nov27_20-35-12_209-20-157-49/events.out.tfevents.1701131664.209-20-157-49.941709.1 +3 -0
  13. special_tokens_map.json +30 -0
  14. tokenizer.json +0 -0
  15. tokenizer_config.json +41 -0
  16. train_results.json +8 -0
  17. trainer_state.json +0 -0
  18. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,134 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: alignment-handbook/zephyr-7b-sft-full
4
+ tags:
5
+ - generated_from_trainer
6
+ model-index:
7
+ - name: zephyr-7b-dpo-full
8
+ results: []
9
+ ---
10
+
11
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
+ should probably proofread and complete it, then remove this comment. -->
13
+
14
+ # zephyr-7b-dpo-full
15
+
16
+ This model is a fine-tuned version of [alignment-handbook/zephyr-7b-sft-full](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full) on an unknown dataset.
17
+ It achieves the following results on the evaluation set:
18
+ - Loss: 0.0254
19
+ - Rewards/chosen: -6.9910
20
+ - Rewards/rejected: -27.5001
21
+ - Rewards/accuracies: 0.9806
22
+ - Rewards/margins: 20.5091
23
+ - Logps/rejected: -345.1052
24
+ - Logps/chosen: -161.1468
25
+ - Logits/rejected: -1.1372
26
+ - Logits/chosen: -1.3943
27
+
28
+ ## Model description
29
+
30
+ More information needed
31
+
32
+ ## Intended uses & limitations
33
+
34
+ More information needed
35
+
36
+ ## Training and evaluation data
37
+
38
+ More information needed
39
+
40
+ ## Training procedure
41
+
42
+ ### Training hyperparameters
43
+
44
+ The following hyperparameters were used during training:
45
+ - learning_rate: 5e-07
46
+ - train_batch_size: 8
47
+ - eval_batch_size: 4
48
+ - seed: 42
49
+ - distributed_type: multi-GPU
50
+ - num_devices: 8
51
+ - total_train_batch_size: 64
52
+ - total_eval_batch_size: 32
53
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
54
+ - lr_scheduler_type: linear
55
+ - lr_scheduler_warmup_ratio: 0.1
56
+ - num_epochs: 3
57
+
58
+ ### Training results
59
+
60
+ | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
61
+ |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
62
+ | 0.3601 | 0.05 | 100 | 0.3410 | -0.0714 | -1.0669 | 0.9444 | 0.9955 | -80.7736 | -91.9507 | -3.0201 | -3.0686 |
63
+ | 0.113 | 0.09 | 200 | 0.1171 | -1.2736 | -4.8848 | 0.9611 | 3.6112 | -118.9524 | -103.9727 | -3.0104 | -3.0297 |
64
+ | 0.0734 | 0.14 | 300 | 0.0768 | -2.2058 | -7.2581 | 0.9778 | 5.0523 | -142.6854 | -113.2948 | -2.9044 | -2.9330 |
65
+ | 0.0587 | 0.18 | 400 | 0.0559 | -4.0388 | -11.7401 | 0.9694 | 7.7012 | -187.5052 | -131.6255 | -2.6966 | -2.7279 |
66
+ | 0.0379 | 0.23 | 500 | 0.0460 | -3.9501 | -12.2735 | 0.9750 | 8.3233 | -192.8393 | -130.7384 | -2.6613 | -2.7017 |
67
+ | 0.0394 | 0.27 | 600 | 0.0405 | -4.8526 | -14.4947 | 0.9750 | 9.6420 | -215.0514 | -139.7636 | -2.3901 | -2.4786 |
68
+ | 0.0375 | 0.32 | 700 | 0.0376 | -3.7095 | -12.6204 | 0.9750 | 8.9110 | -196.3089 | -128.3319 | -2.6708 | -2.7142 |
69
+ | 0.043 | 0.37 | 800 | 0.0375 | -5.2652 | -14.8540 | 0.9694 | 9.5888 | -218.6447 | -143.8889 | -2.1654 | -2.2829 |
70
+ | 0.0304 | 0.41 | 900 | 0.0380 | -4.2196 | -13.7402 | 0.9750 | 9.5207 | -207.5071 | -133.4332 | -2.1041 | -2.2015 |
71
+ | 0.0254 | 0.46 | 1000 | 0.0324 | -4.9741 | -16.1886 | 0.9722 | 11.2145 | -231.9906 | -140.9781 | -2.0860 | -2.1849 |
72
+ | 0.03 | 0.5 | 1100 | 0.0342 | -5.1530 | -16.4077 | 0.9667 | 11.2547 | -234.1812 | -142.7671 | -2.1132 | -2.1749 |
73
+ | 0.0339 | 0.55 | 1200 | 0.0311 | -2.7641 | -11.9997 | 0.9750 | 9.2356 | -190.1020 | -118.8785 | -2.0003 | -2.1125 |
74
+ | 0.0489 | 0.59 | 1300 | 0.0272 | -2.8374 | -12.4331 | 0.9833 | 9.5957 | -194.4359 | -119.6116 | -2.1739 | -2.2572 |
75
+ | 0.0263 | 0.64 | 1400 | 0.0291 | -3.4940 | -13.2533 | 0.9833 | 9.7593 | -202.6373 | -126.1769 | -2.0814 | -2.2103 |
76
+ | 0.0301 | 0.68 | 1500 | 0.0266 | -4.7000 | -16.2144 | 0.9778 | 11.5144 | -232.2482 | -138.2372 | -2.0580 | -2.1682 |
77
+ | 0.0272 | 0.73 | 1600 | 0.0283 | -6.1020 | -18.1286 | 0.9667 | 12.0266 | -251.3911 | -152.2577 | -1.8091 | -1.9654 |
78
+ | 0.0278 | 0.78 | 1700 | 0.0254 | -2.9913 | -13.0756 | 0.9750 | 10.0843 | -200.8606 | -121.1501 | -2.1657 | -2.2443 |
79
+ | 0.0291 | 0.82 | 1800 | 0.0260 | -4.8989 | -16.4318 | 0.9722 | 11.5329 | -234.4226 | -140.2258 | -1.9427 | -2.0364 |
80
+ | 0.0253 | 0.87 | 1900 | 0.0252 | -4.0403 | -15.3217 | 0.9778 | 11.2813 | -223.3212 | -131.6404 | -1.9099 | -2.0361 |
81
+ | 0.0235 | 0.91 | 2000 | 0.0223 | -3.1973 | -14.8592 | 0.9750 | 11.6619 | -218.6964 | -123.2101 | -2.0448 | -2.1573 |
82
+ | 0.0272 | 0.96 | 2100 | 0.0236 | -3.5828 | -15.3281 | 0.9750 | 11.7453 | -223.3855 | -127.0649 | -1.9834 | -2.0892 |
83
+ | 0.0107 | 1.0 | 2200 | 0.0206 | -2.6028 | -14.3743 | 0.9806 | 11.7715 | -213.8473 | -117.2654 | -2.1607 | -2.2658 |
84
+ | 0.0089 | 1.05 | 2300 | 0.0209 | -4.2378 | -18.4889 | 0.9806 | 14.2511 | -254.9935 | -133.6152 | -1.8951 | -2.0551 |
85
+ | 0.0103 | 1.1 | 2400 | 0.0222 | -3.8267 | -16.7762 | 0.9778 | 12.9495 | -237.8671 | -129.5042 | -2.1516 | -2.2577 |
86
+ | 0.0114 | 1.14 | 2500 | 0.0230 | -5.6573 | -20.9027 | 0.9750 | 15.2454 | -279.1315 | -147.8097 | -1.6603 | -1.8552 |
87
+ | 0.0075 | 1.19 | 2600 | 0.0217 | -4.9252 | -19.3893 | 0.9833 | 14.4641 | -263.9972 | -140.4892 | -1.8500 | -2.0172 |
88
+ | 0.0084 | 1.23 | 2700 | 0.0235 | -5.3971 | -20.1630 | 0.9806 | 14.7660 | -271.7348 | -145.2077 | -1.8673 | -2.0178 |
89
+ | 0.0094 | 1.28 | 2800 | 0.0253 | -4.4422 | -18.2275 | 0.9778 | 13.7853 | -252.3796 | -135.6592 | -2.0417 | -2.1522 |
90
+ | 0.0064 | 1.32 | 2900 | 0.0269 | -4.3177 | -18.7842 | 0.9750 | 14.4665 | -257.9463 | -134.4144 | -2.0273 | -2.1416 |
91
+ | 0.0093 | 1.37 | 3000 | 0.0234 | -4.6335 | -19.3959 | 0.9722 | 14.7624 | -264.0636 | -137.5719 | -1.9133 | -2.0781 |
92
+ | 0.0083 | 1.41 | 3100 | 0.0230 | -5.0798 | -20.1230 | 0.9806 | 15.0431 | -271.3341 | -142.0356 | -1.7624 | -1.9098 |
93
+ | 0.0108 | 1.46 | 3200 | 0.0217 | -3.8920 | -18.1320 | 0.9806 | 14.2400 | -251.4249 | -130.1573 | -1.9500 | -2.0745 |
94
+ | 0.0091 | 1.51 | 3300 | 0.0223 | -5.6969 | -21.8594 | 0.9806 | 16.1624 | -288.6983 | -148.2067 | -1.9011 | -2.0357 |
95
+ | 0.012 | 1.55 | 3400 | 0.0215 | -4.8539 | -18.2010 | 0.9861 | 13.3471 | -252.1147 | -139.7764 | -1.7467 | -1.8729 |
96
+ | 0.0064 | 1.6 | 3500 | 0.0218 | -5.0782 | -19.6405 | 0.9833 | 14.5623 | -266.5100 | -142.0197 | -1.7721 | -1.8976 |
97
+ | 0.0078 | 1.64 | 3600 | 0.0213 | -5.1302 | -20.2433 | 0.9778 | 15.1130 | -272.5372 | -142.5395 | -1.5605 | -1.7238 |
98
+ | 0.0076 | 1.69 | 3700 | 0.0222 | -5.6579 | -22.5669 | 0.9778 | 16.9089 | -295.7735 | -147.8167 | -1.4705 | -1.6559 |
99
+ | 0.0124 | 1.73 | 3800 | 0.0221 | -3.5477 | -16.1322 | 0.9861 | 12.5845 | -231.4264 | -126.7141 | -1.6382 | -1.7606 |
100
+ | 0.0065 | 1.78 | 3900 | 0.0201 | -5.0615 | -20.0072 | 0.9806 | 14.9457 | -270.1767 | -141.8525 | -1.5657 | -1.7132 |
101
+ | 0.0078 | 1.83 | 4000 | 0.0200 | -5.5288 | -20.5404 | 0.9806 | 15.0115 | -275.5083 | -146.5255 | -1.6180 | -1.7627 |
102
+ | 0.0089 | 1.87 | 4100 | 0.0210 | -5.5847 | -22.2109 | 0.9806 | 16.6262 | -292.2132 | -147.0840 | -1.6373 | -1.7998 |
103
+ | 0.0022 | 1.92 | 4200 | 0.0206 | -5.2113 | -21.4282 | 0.9833 | 16.2169 | -284.3861 | -143.3500 | -1.5687 | -1.7379 |
104
+ | 0.0065 | 1.96 | 4300 | 0.0205 | -4.4336 | -18.8360 | 0.9806 | 14.4024 | -258.4650 | -135.5732 | -1.5889 | -1.7431 |
105
+ | 0.0038 | 2.01 | 4400 | 0.0213 | -3.8031 | -17.3943 | 0.9833 | 13.5912 | -244.0473 | -129.2682 | -1.6138 | -1.7573 |
106
+ | 0.0042 | 2.05 | 4500 | 0.0210 | -4.7797 | -20.6519 | 0.9833 | 15.8722 | -276.6233 | -139.0337 | -1.4571 | -1.6393 |
107
+ | 0.0025 | 2.1 | 4600 | 0.0220 | -5.3030 | -22.1802 | 0.9833 | 16.8772 | -291.9064 | -144.2672 | -1.4136 | -1.6114 |
108
+ | 0.0013 | 2.15 | 4700 | 0.0240 | -6.9006 | -26.5257 | 0.9806 | 19.6251 | -335.3617 | -160.2434 | -1.3102 | -1.5345 |
109
+ | 0.0075 | 2.19 | 4800 | 0.0253 | -6.6613 | -26.8086 | 0.9778 | 20.1473 | -338.1903 | -157.8497 | -1.2594 | -1.4933 |
110
+ | 0.0024 | 2.24 | 4900 | 0.0238 | -5.9384 | -25.2963 | 0.9806 | 19.3579 | -323.0674 | -150.6208 | -1.3039 | -1.5318 |
111
+ | 0.0017 | 2.28 | 5000 | 0.0217 | -5.7146 | -23.2521 | 0.9806 | 17.5375 | -302.6257 | -148.3833 | -1.3104 | -1.5277 |
112
+ | 0.0037 | 2.33 | 5100 | 0.0234 | -6.8711 | -26.8591 | 0.9806 | 19.9880 | -338.6961 | -159.9486 | -1.2368 | -1.4798 |
113
+ | 0.0045 | 2.37 | 5200 | 0.0233 | -6.4563 | -25.4117 | 0.9806 | 18.9554 | -324.2215 | -155.7999 | -1.2791 | -1.5095 |
114
+ | 0.0024 | 2.42 | 5300 | 0.0226 | -6.0528 | -23.6960 | 0.9806 | 17.6432 | -307.0642 | -151.7647 | -1.2664 | -1.4908 |
115
+ | 0.0023 | 2.46 | 5400 | 0.0233 | -6.6659 | -25.6664 | 0.9806 | 19.0004 | -326.7684 | -157.8966 | -1.2361 | -1.4788 |
116
+ | 0.0007 | 2.51 | 5500 | 0.0250 | -6.7756 | -26.1454 | 0.9806 | 19.3698 | -331.5591 | -158.9936 | -1.2610 | -1.4948 |
117
+ | 0.0029 | 2.56 | 5600 | 0.0246 | -6.6468 | -25.9933 | 0.9806 | 19.3465 | -330.0380 | -157.7054 | -1.3105 | -1.5473 |
118
+ | 0.0017 | 2.6 | 5700 | 0.0248 | -6.9095 | -26.0328 | 0.9806 | 19.1234 | -330.4331 | -160.3320 | -1.2445 | -1.4880 |
119
+ | 0.0055 | 2.65 | 5800 | 0.0257 | -7.5666 | -27.8213 | 0.9806 | 20.2547 | -348.3180 | -166.9033 | -1.1526 | -1.4106 |
120
+ | 0.0036 | 2.69 | 5900 | 0.0263 | -7.6427 | -28.5222 | 0.9806 | 20.8795 | -355.3266 | -167.6644 | -1.0877 | -1.3563 |
121
+ | 0.0061 | 2.74 | 6000 | 0.0242 | -6.7938 | -26.3053 | 0.9806 | 19.5115 | -333.1572 | -159.1749 | -1.2176 | -1.4633 |
122
+ | 0.0018 | 2.78 | 6100 | 0.0242 | -6.8675 | -26.5397 | 0.9806 | 19.6722 | -335.5020 | -159.9126 | -1.1958 | -1.4429 |
123
+ | 0.0024 | 2.83 | 6200 | 0.0249 | -7.1136 | -27.2224 | 0.9806 | 20.1088 | -342.3287 | -162.3729 | -1.1466 | -1.4003 |
124
+ | 0.0026 | 2.88 | 6300 | 0.0251 | -6.9595 | -27.0923 | 0.9806 | 20.1328 | -341.0272 | -160.8322 | -1.1501 | -1.4025 |
125
+ | 0.0034 | 2.92 | 6400 | 0.0253 | -7.1263 | -27.7050 | 0.9806 | 20.5787 | -347.1544 | -162.5005 | -1.1288 | -1.3867 |
126
+ | 0.0023 | 2.97 | 6500 | 0.0253 | -7.0828 | -27.6979 | 0.9806 | 20.6151 | -347.0838 | -162.0652 | -1.1284 | -1.3867 |
127
+
128
+
129
+ ### Framework versions
130
+
131
+ - Transformers 4.35.0
132
+ - Pytorch 2.1.1+cu121
133
+ - Datasets 2.14.6
134
+ - Tokenizers 0.14.1
all_results.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "eval_logits/chosen": -1.3942855596542358,
4
+ "eval_logits/rejected": -1.1372073888778687,
5
+ "eval_logps/chosen": -161.14675903320312,
6
+ "eval_logps/rejected": -345.1051940917969,
7
+ "eval_loss": 0.02538341097533703,
8
+ "eval_rewards/accuracies": 0.980555534362793,
9
+ "eval_rewards/chosen": -6.990954399108887,
10
+ "eval_rewards/margins": 20.509109497070312,
11
+ "eval_rewards/rejected": -27.500062942504883,
12
+ "eval_runtime": 49.4371,
13
+ "eval_samples": 2862,
14
+ "eval_samples_per_second": 57.892,
15
+ "eval_steps_per_second": 1.82,
16
+ "train_loss": 0.026962732661432107,
17
+ "train_runtime": 14195.2291,
18
+ "train_samples": 140201,
19
+ "train_samples_per_second": 29.63,
20
+ "train_steps_per_second": 0.463
21
+ }
config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "alignment-handbook/zephyr-7b-sft-full",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "bos_token_id": 1,
7
+ "eos_token_id": 2,
8
+ "hidden_act": "silu",
9
+ "hidden_size": 4096,
10
+ "initializer_range": 0.02,
11
+ "intermediate_size": 14336,
12
+ "max_position_embeddings": 32768,
13
+ "model_type": "mistral",
14
+ "num_attention_heads": 32,
15
+ "num_hidden_layers": 32,
16
+ "num_key_value_heads": 8,
17
+ "rms_norm_eps": 1e-05,
18
+ "rope_theta": 10000.0,
19
+ "sliding_window": 4096,
20
+ "tie_word_embeddings": false,
21
+ "torch_dtype": "bfloat16",
22
+ "transformers_version": "4.35.0",
23
+ "use_cache": false,
24
+ "vocab_size": 32000
25
+ }
eval_results.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "eval_logits/chosen": -1.3942855596542358,
4
+ "eval_logits/rejected": -1.1372073888778687,
5
+ "eval_logps/chosen": -161.14675903320312,
6
+ "eval_logps/rejected": -345.1051940917969,
7
+ "eval_loss": 0.02538341097533703,
8
+ "eval_rewards/accuracies": 0.980555534362793,
9
+ "eval_rewards/chosen": -6.990954399108887,
10
+ "eval_rewards/margins": 20.509109497070312,
11
+ "eval_rewards/rejected": -27.500062942504883,
12
+ "eval_runtime": 49.4371,
13
+ "eval_samples": 2862,
14
+ "eval_samples_per_second": 57.892,
15
+ "eval_steps_per_second": 1.82
16
+ }
generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 2,
5
+ "transformers_version": "4.35.0"
6
+ }
model-00001-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e6014241227174be611ecad7f56b22da7339854d5ae2a4e5f4702c3cafb160cb
3
+ size 4943162336
model-00002-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9542a6cdad7252f09e1727e18f0e455574cd1d924de84d0b24d69c216e54e2e1
3
+ size 4999819336
model-00003-of-00003.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65020de2b2aef18485c92ded6193a47cfccf36dd22a69d255eda078f88d83e7d
3
+ size 4540516344
model.safetensors.index.json ADDED
@@ -0,0 +1,298 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 14483464192
4
+ },
5
+ "weight_map": {
6
+ "lm_head.weight": "model-00003-of-00003.safetensors",
7
+ "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
8
+ "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
9
+ "model.layers.0.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
10
+ "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
11
+ "model.layers.0.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
12
+ "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
13
+ "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
14
+ "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
15
+ "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
16
+ "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
17
+ "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
18
+ "model.layers.1.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
19
+ "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
20
+ "model.layers.1.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
21
+ "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
22
+ "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
23
+ "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
24
+ "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
25
+ "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
26
+ "model.layers.10.input_layernorm.weight": "model-00002-of-00003.safetensors",
27
+ "model.layers.10.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
28
+ "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
29
+ "model.layers.10.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
30
+ "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
31
+ "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
32
+ "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
33
+ "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
34
+ "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
35
+ "model.layers.11.input_layernorm.weight": "model-00002-of-00003.safetensors",
36
+ "model.layers.11.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
37
+ "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
38
+ "model.layers.11.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
39
+ "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
40
+ "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
41
+ "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
42
+ "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
43
+ "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
44
+ "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
45
+ "model.layers.12.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
46
+ "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
47
+ "model.layers.12.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
48
+ "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
49
+ "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
50
+ "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
51
+ "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
52
+ "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
53
+ "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
54
+ "model.layers.13.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
55
+ "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
56
+ "model.layers.13.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
57
+ "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
58
+ "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
59
+ "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
60
+ "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
61
+ "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
62
+ "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
63
+ "model.layers.14.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
64
+ "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
65
+ "model.layers.14.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
66
+ "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
67
+ "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
68
+ "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
69
+ "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
70
+ "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
71
+ "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
72
+ "model.layers.15.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
73
+ "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
74
+ "model.layers.15.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
75
+ "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
76
+ "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
77
+ "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
78
+ "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
79
+ "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
80
+ "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
81
+ "model.layers.16.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
82
+ "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
83
+ "model.layers.16.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
84
+ "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
85
+ "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
86
+ "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
87
+ "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
88
+ "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
89
+ "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
90
+ "model.layers.17.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
91
+ "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
92
+ "model.layers.17.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
93
+ "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
94
+ "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
95
+ "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
96
+ "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
97
+ "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
98
+ "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
99
+ "model.layers.18.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
100
+ "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
101
+ "model.layers.18.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
102
+ "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
103
+ "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
104
+ "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
105
+ "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
106
+ "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
107
+ "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
108
+ "model.layers.19.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
109
+ "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
110
+ "model.layers.19.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
111
+ "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
112
+ "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
113
+ "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
114
+ "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
115
+ "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
116
+ "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
117
+ "model.layers.2.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
118
+ "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
119
+ "model.layers.2.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
120
+ "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
121
+ "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
122
+ "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
123
+ "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
124
+ "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
125
+ "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
126
+ "model.layers.20.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
127
+ "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
128
+ "model.layers.20.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
129
+ "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
130
+ "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
131
+ "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
132
+ "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
133
+ "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
134
+ "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
135
+ "model.layers.21.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
136
+ "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
137
+ "model.layers.21.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
138
+ "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
139
+ "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
140
+ "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
141
+ "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
142
+ "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
143
+ "model.layers.22.input_layernorm.weight": "model-00003-of-00003.safetensors",
144
+ "model.layers.22.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
145
+ "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
146
+ "model.layers.22.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
147
+ "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
148
+ "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
149
+ "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
150
+ "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
151
+ "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
152
+ "model.layers.23.input_layernorm.weight": "model-00003-of-00003.safetensors",
153
+ "model.layers.23.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
154
+ "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
155
+ "model.layers.23.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
156
+ "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
157
+ "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
158
+ "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
159
+ "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
160
+ "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
161
+ "model.layers.24.input_layernorm.weight": "model-00003-of-00003.safetensors",
162
+ "model.layers.24.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
163
+ "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
164
+ "model.layers.24.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
165
+ "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
166
+ "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
167
+ "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
168
+ "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
169
+ "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
170
+ "model.layers.25.input_layernorm.weight": "model-00003-of-00003.safetensors",
171
+ "model.layers.25.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
172
+ "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
173
+ "model.layers.25.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
174
+ "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
175
+ "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
176
+ "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
177
+ "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
178
+ "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
179
+ "model.layers.26.input_layernorm.weight": "model-00003-of-00003.safetensors",
180
+ "model.layers.26.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
181
+ "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
182
+ "model.layers.26.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
183
+ "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
184
+ "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
185
+ "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
186
+ "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
187
+ "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
188
+ "model.layers.27.input_layernorm.weight": "model-00003-of-00003.safetensors",
189
+ "model.layers.27.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
190
+ "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
191
+ "model.layers.27.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
192
+ "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
193
+ "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
194
+ "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
195
+ "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
196
+ "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
197
+ "model.layers.28.input_layernorm.weight": "model-00003-of-00003.safetensors",
198
+ "model.layers.28.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
199
+ "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
200
+ "model.layers.28.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
201
+ "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
202
+ "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
203
+ "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
204
+ "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
205
+ "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
206
+ "model.layers.29.input_layernorm.weight": "model-00003-of-00003.safetensors",
207
+ "model.layers.29.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
208
+ "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
209
+ "model.layers.29.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
210
+ "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
211
+ "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
212
+ "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
213
+ "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
214
+ "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
215
+ "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
216
+ "model.layers.3.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
217
+ "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
218
+ "model.layers.3.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
219
+ "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
220
+ "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
221
+ "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
222
+ "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
223
+ "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
224
+ "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
225
+ "model.layers.30.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
226
+ "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
227
+ "model.layers.30.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
228
+ "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
229
+ "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
230
+ "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
231
+ "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
232
+ "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
233
+ "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
234
+ "model.layers.31.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
235
+ "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
236
+ "model.layers.31.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
237
+ "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
238
+ "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
239
+ "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
240
+ "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
241
+ "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
242
+ "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
243
+ "model.layers.4.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
244
+ "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
245
+ "model.layers.4.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
246
+ "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
247
+ "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
248
+ "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
249
+ "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
250
+ "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
251
+ "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
252
+ "model.layers.5.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
253
+ "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
254
+ "model.layers.5.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
255
+ "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
256
+ "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
257
+ "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
258
+ "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
259
+ "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
260
+ "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
261
+ "model.layers.6.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
262
+ "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
263
+ "model.layers.6.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
264
+ "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
265
+ "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
266
+ "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
267
+ "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
268
+ "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
269
+ "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
270
+ "model.layers.7.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
271
+ "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
272
+ "model.layers.7.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
273
+ "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
274
+ "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
275
+ "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
276
+ "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
277
+ "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
278
+ "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
279
+ "model.layers.8.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
280
+ "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
281
+ "model.layers.8.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
282
+ "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
283
+ "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
284
+ "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
285
+ "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
286
+ "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
287
+ "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
288
+ "model.layers.9.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
289
+ "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
290
+ "model.layers.9.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
291
+ "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
292
+ "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
293
+ "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
294
+ "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
295
+ "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
296
+ "model.norm.weight": "model-00003-of-00003.safetensors"
297
+ }
298
+ }
runs/Nov27_20-27-33_209-20-157-49/events.out.tfevents.1701116942.209-20-157-49.939604.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac768e62be9300ec030338dfc762be54b6b42f8e0053a186bae3aaac0b67d9a0
3
+ size 4283
runs/Nov27_20-35-12_209-20-157-49/events.out.tfevents.1701117420.209-20-157-49.941709.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52480f6f43fba4e9d3404b0bb11c21be74b74730ad027858a03184aca7274022
3
+ size 469753
runs/Nov27_20-35-12_209-20-157-49/events.out.tfevents.1701131664.209-20-157-49.941709.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:defb12841da07e0d5e51ee1e6f3768ec4376a50da50ef62de82bd753d405d74e
3
+ size 828
special_tokens_map.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "unk_token": {
24
+ "content": "<unk>",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ }
30
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<unk>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<s>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ }
27
+ },
28
+ "additional_special_tokens": [],
29
+ "bos_token": "<s>",
30
+ "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n' + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
31
+ "clean_up_tokenization_spaces": false,
32
+ "eos_token": "</s>",
33
+ "legacy": true,
34
+ "model_max_length": 2048,
35
+ "pad_token": "</s>",
36
+ "sp_model_kwargs": {},
37
+ "spaces_between_special_tokens": false,
38
+ "tokenizer_class": "LlamaTokenizer",
39
+ "unk_token": "<unk>",
40
+ "use_default_system_prompt": true
41
+ }
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 3.0,
3
+ "train_loss": 0.026962732661432107,
4
+ "train_runtime": 14195.2291,
5
+ "train_samples": 140201,
6
+ "train_samples_per_second": 29.63,
7
+ "train_steps_per_second": 0.463
8
+ }
trainer_state.json ADDED
The diff for this file is too large to render. See raw diff
 
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2fb403f5ed158a0cac23483ad56863aaa0e5dbda2999cfb2059fab104737c508
3
+ size 5688