Hugging Face's logo

llama-duo
/
gemma7b-summarize-gpt4o-32k

PEFT
TensorBoard
Safetensors
gemma
alignment-handbook
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Metrics Community
Kat380 commited on
Commit
ebc790b
•
1 Parent(s): 9b4b7c1

update model checkpoints

Browse files
Files changed (11) hide show
  1. README.md +19 -22
  2. adapter_config.json +4 -9
  3. adapter_model.safetensors +2 -2
  4. all_results.json +11 -11
  5. config.json +3 -3
  6. eval_results.json +5 -5
  7. runs/Jun13_06-11-24_gpu1-2/events.out.tfevents.1718230372.gpu1-2.1135362.0 +3 -0
  8. runs/Jun13_06-11-24_gpu1-2/events.out.tfevents.1718231704.gpu1-2.1135362.1 +3 -0
  9. train_results.json +7 -7
  10. trainer_state.json +533 -2605
  11. training_args.bin +1 -1
README.md CHANGED
@@ -1,14 +1,11 @@
1
  ---
2
- license: gemma
3
  library_name: peft
4
  tags:
5
  - alignment-handbook
6
- - trl
7
- - sft
8
  - generated_from_trainer
9
- base_model: google/gemma-7b
10
  datasets:
11
  - llama-duo/synth_summarize_dataset_dedup
 
12
  model-index:
13
  - name: gemma7b-summarize-gpt4o-32k
14
  results: []
@@ -21,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
21
 
22
  This model is a fine-tuned version of [google/gemma-7b](https://huggingface.co/google/gemma-7b) on the llama-duo/synth_summarize_dataset_dedup dataset.
23
  It achieves the following results on the evaluation set:
24
- - Loss: 4.7170
25
 
26
  ## Model description
27
 
@@ -42,13 +39,13 @@ More information needed
42
  The following hyperparameters were used during training:
43
  - learning_rate: 0.0002
44
  - train_batch_size: 4
45
- - eval_batch_size: 4
46
  - seed: 42
47
  - distributed_type: multi-GPU
48
- - num_devices: 2
49
  - gradient_accumulation_steps: 2
50
- - total_train_batch_size: 16
51
- - total_eval_batch_size: 8
52
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
53
  - lr_scheduler_type: cosine
54
  - lr_scheduler_warmup_ratio: 0.1
@@ -56,24 +53,24 @@ The following hyperparameters were used during training:
56
 
57
  ### Training results
58
 
59
- | Training Loss | Epoch | Step | Validation Loss |
60
- |:-------------:|:------:|:----:|:---------------:|
61
- | 1.0031 | 0.9975 | 203 | 2.5299 |
62
- | 0.8685 | 2.0 | 407 | 2.4260 |
63
- | 0.8 | 2.9975 | 610 | 2.5051 |
64
- | 0.6938 | 4.0 | 814 | 2.6558 |
65
- | 0.5865 | 4.9975 | 1017 | 2.9163 |
66
- | 0.4821 | 6.0 | 1221 | 3.3285 |
67
- | 0.3899 | 6.9975 | 1424 | 3.8984 |
68
- | 0.3321 | 8.0 | 1628 | 4.4348 |
69
- | 0.3089 | 8.9975 | 1831 | 4.6891 |
70
- | 0.3016 | 9.9754 | 2030 | 4.7170 |
71
 
72
 
73
  ### Framework versions
74
 
75
  - PEFT 0.10.0
76
  - Transformers 4.40.0
77
- - Pytorch 2.2.1+cu121
78
  - Datasets 2.18.0
79
  - Tokenizers 0.19.1
 
1
  ---
 
2
  library_name: peft
3
  tags:
4
  - alignment-handbook
 
 
5
  - generated_from_trainer
 
6
  datasets:
7
  - llama-duo/synth_summarize_dataset_dedup
8
+ base_model: google/gemma-7b
9
  model-index:
10
  - name: gemma7b-summarize-gpt4o-32k
11
  results: []
 
18
 
19
  This model is a fine-tuned version of [google/gemma-7b](https://huggingface.co/google/gemma-7b) on the llama-duo/synth_summarize_dataset_dedup dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 2.5602
22
 
23
  ## Model description
24
 
 
39
  The following hyperparameters were used during training:
40
  - learning_rate: 0.0002
41
  - train_batch_size: 4
42
+ - eval_batch_size: 2
43
  - seed: 42
44
  - distributed_type: multi-GPU
45
+ - num_devices: 8
46
  - gradient_accumulation_steps: 2
47
+ - total_train_batch_size: 64
48
+ - total_eval_batch_size: 16
49
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
50
  - lr_scheduler_type: cosine
51
  - lr_scheduler_warmup_ratio: 0.1
 
53
 
54
  ### Training results
55
 
56
+ | Training Loss | Epoch | Step | Validation Loss |
57
+ |:-------------:|:-----:|:----:|:---------------:|
58
+ | 3.5798 | 1.0 | 55 | 4.0590 |
59
+ | 1.346 | 2.0 | 110 | 2.6944 |
60
+ | 1.1944 | 3.0 | 165 | 2.6027 |
61
+ | 1.1119 | 4.0 | 220 | 2.5779 |
62
+ | 1.0741 | 5.0 | 275 | 2.5495 |
63
+ | 1.0435 | 6.0 | 330 | 2.5501 |
64
+ | 1.0191 | 7.0 | 385 | 2.5536 |
65
+ | 0.9965 | 8.0 | 440 | 2.5604 |
66
+ | 0.9986 | 9.0 | 495 | 2.5597 |
67
+ | 0.9948 | 10.0 | 550 | 2.5602 |
68
 
69
 
70
  ### Framework versions
71
 
72
  - PEFT 0.10.0
73
  - Transformers 4.40.0
74
+ - Pytorch 2.1.2+cu121
75
  - Datasets 2.18.0
76
  - Tokenizers 0.19.1
adapter_config.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "alpha_pattern": {},
3
  "auto_mapping": null,
4
- "base_model_name_or_path": "google/gemma-7b",
5
  "bias": "none",
6
  "fan_in_fan_out": false,
7
  "inference_mode": true,
@@ -10,23 +10,18 @@
10
  "layers_pattern": null,
11
  "layers_to_transform": null,
12
  "loftq_config": {},
13
- "lora_alpha": 16,
14
  "lora_dropout": 0.05,
15
  "megatron_config": null,
16
  "megatron_core": "megatron.core",
17
  "modules_to_save": null,
18
  "peft_type": "LORA",
19
- "r": 8,
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
  "v_proj",
24
- "k_proj",
25
- "o_proj",
26
- "down_proj",
27
- "up_proj",
28
- "q_proj",
29
- "gate_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
1
  {
2
  "alpha_pattern": {},
3
  "auto_mapping": null,
4
+ "base_model_name_or_path": "/hpc2hdd/home/fwang380/OpenSource/Models/gemma-7b",
5
  "bias": "none",
6
  "fan_in_fan_out": false,
7
  "inference_mode": true,
 
10
  "layers_pattern": null,
11
  "layers_to_transform": null,
12
  "loftq_config": {},
13
+ "lora_alpha": 32,
14
  "lora_dropout": 0.05,
15
  "megatron_config": null,
16
  "megatron_core": "megatron.core",
17
  "modules_to_save": null,
18
  "peft_type": "LORA",
19
+ "r": 16,
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
  "v_proj",
24
+ "q_proj"
 
 
 
 
 
25
  ],
26
  "task_type": "CAUSAL_LM",
27
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:125f331ca9a8684aec05b7fd7ab37fb0f3ec43d64414a7c4aa9dfc598b180031
3
- size 50056096
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bf418905f7800a9c225c813c0bcb18b5fabd4cf79b2b3fb4b806ccb3e38eb8f
3
+ size 12860096
all_results.json CHANGED
@@ -1,14 +1,14 @@
1
  {
2
- "epoch": 9.975429975429975,
3
- "eval_loss": 4.717045307159424,
4
- "eval_runtime": 2.0634,
5
  "eval_samples": 25,
6
- "eval_samples_per_second": 4.846,
7
- "eval_steps_per_second": 0.969,
8
- "total_flos": 1.5518062706111283e+18,
9
- "train_loss": 1.3545548074938394,
10
- "train_runtime": 13205.4864,
11
- "train_samples": 29997,
12
- "train_samples_per_second": 2.465,
13
- "train_steps_per_second": 0.154
14
  }
 
1
  {
2
+ "epoch": 10.0,
3
+ "eval_loss": 2.5602283477783203,
4
+ "eval_runtime": 0.2344,
5
  "eval_samples": 25,
6
+ "eval_samples_per_second": 42.666,
7
+ "eval_steps_per_second": 4.267,
8
+ "total_flos": 1.6777423328808796e+18,
9
+ "train_loss": 3.202145513187755,
10
+ "train_runtime": 1331.6624,
11
+ "train_samples": 32305,
12
+ "train_samples_per_second": 26.313,
13
+ "train_steps_per_second": 0.413
14
  }
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "google/gemma-7b",
3
  "architectures": [
4
  "GemmaForCausalLM"
5
  ],
@@ -23,9 +23,9 @@
23
  "_load_in_4bit": true,
24
  "_load_in_8bit": false,
25
  "bnb_4bit_compute_dtype": "bfloat16",
26
- "bnb_4bit_quant_storage": "bfloat16",
27
  "bnb_4bit_quant_type": "nf4",
28
- "bnb_4bit_use_double_quant": true,
29
  "llm_int8_enable_fp32_cpu_offload": false,
30
  "llm_int8_has_fp16_weight": false,
31
  "llm_int8_skip_modules": null,
 
1
  {
2
+ "_name_or_path": "/hpc2hdd/home/fwang380/OpenSource/Models/gemma-7b",
3
  "architectures": [
4
  "GemmaForCausalLM"
5
  ],
 
23
  "_load_in_4bit": true,
24
  "_load_in_8bit": false,
25
  "bnb_4bit_compute_dtype": "bfloat16",
26
+ "bnb_4bit_quant_storage": "uint8",
27
  "bnb_4bit_quant_type": "nf4",
28
+ "bnb_4bit_use_double_quant": false,
29
  "llm_int8_enable_fp32_cpu_offload": false,
30
  "llm_int8_has_fp16_weight": false,
31
  "llm_int8_skip_modules": null,
eval_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
- "epoch": 9.975429975429975,
3
- "eval_loss": 4.717045307159424,
4
- "eval_runtime": 2.0634,
5
  "eval_samples": 25,
6
- "eval_samples_per_second": 4.846,
7
- "eval_steps_per_second": 0.969
8
  }
 
1
  {
2
+ "epoch": 10.0,
3
+ "eval_loss": 2.5602283477783203,
4
+ "eval_runtime": 0.2344,
5
  "eval_samples": 25,
6
+ "eval_samples_per_second": 42.666,
7
+ "eval_steps_per_second": 4.267
8
  }
runs/Jun13_06-11-24_gpu1-2/events.out.tfevents.1718230372.gpu1-2.1135362.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d67de193adcb4d80b1f0b1ae3a9884c43fa718e5a324b60954b7ef2e445aa18
3
+ size 31822
runs/Jun13_06-11-24_gpu1-2/events.out.tfevents.1718231704.gpu1-2.1135362.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54628faf2b0a0a9c2d9de830029cec936a75a4545f08d51e8c5d9000dee67dbc
3
+ size 359
train_results.json CHANGED
@@ -1,9 +1,9 @@
1
  {
2
- "epoch": 9.975429975429975,
3
- "total_flos": 1.5518062706111283e+18,
4
- "train_loss": 1.3545548074938394,
5
- "train_runtime": 13205.4864,
6
- "train_samples": 29997,
7
- "train_samples_per_second": 2.465,
8
- "train_steps_per_second": 0.154
9
  }
 
1
  {
2
+ "epoch": 10.0,
3
+ "total_flos": 1.6777423328808796e+18,
4
+ "train_loss": 3.202145513187755,
5
+ "train_runtime": 1331.6624,
6
+ "train_samples": 32305,
7
+ "train_samples_per_second": 26.313,
8
+ "train_steps_per_second": 0.413
9
  }
trainer_state.json CHANGED
@@ -1,2958 +1,886 @@
1
  {
2
  "best_metric": null,
3
  "best_model_checkpoint": null,
4
- "epoch": 9.975429975429975,
5
  "eval_steps": 500,
6
- "global_step": 2030,
7
  "is_hyper_param_search": false,
8
  "is_local_process_zero": true,
9
  "is_world_process_zero": true,
10
  "log_history": [
11
  {
12
- "epoch": 0.004914004914004914,
13
- "grad_norm": 552.0,
14
- "learning_rate": 9.852216748768474e-07,
15
- "loss": 51.7129,
16
  "step": 1
17
  },
18
  {
19
- "epoch": 0.02457002457002457,
20
- "grad_norm": 404.0,
21
- "learning_rate": 4.926108374384237e-06,
22
- "loss": 45.7069,
23
  "step": 5
24
  },
25
  {
26
- "epoch": 0.04914004914004914,
27
- "grad_norm": 388.0,
28
- "learning_rate": 9.852216748768475e-06,
29
- "loss": 46.9702,
30
  "step": 10
31
  },
32
  {
33
- "epoch": 0.07371007371007371,
34
- "grad_norm": 292.0,
35
- "learning_rate": 1.4778325123152711e-05,
36
- "loss": 37.466,
37
  "step": 15
38
  },
39
  {
40
- "epoch": 0.09828009828009827,
41
- "grad_norm": 79.5,
42
- "learning_rate": 1.970443349753695e-05,
43
- "loss": 28.4521,
44
  "step": 20
45
  },
46
  {
47
- "epoch": 0.12285012285012285,
48
- "grad_norm": 47.25,
49
- "learning_rate": 2.4630541871921184e-05,
50
- "loss": 23.8617,
51
  "step": 25
52
  },
53
  {
54
- "epoch": 0.14742014742014742,
55
- "grad_norm": 26.625,
56
- "learning_rate": 2.9556650246305422e-05,
57
- "loss": 22.4994,
58
  "step": 30
59
  },
60
  {
61
- "epoch": 0.171990171990172,
62
- "grad_norm": 12.625,
63
- "learning_rate": 3.4482758620689657e-05,
64
- "loss": 20.8855,
65
  "step": 35
66
  },
67
  {
68
- "epoch": 0.19656019656019655,
69
- "grad_norm": 7.84375,
70
- "learning_rate": 3.94088669950739e-05,
71
- "loss": 19.2048,
72
  "step": 40
73
  },
74
  {
75
- "epoch": 0.22113022113022113,
76
- "grad_norm": 8.5,
77
- "learning_rate": 4.433497536945813e-05,
78
- "loss": 18.9068,
79
  "step": 45
80
  },
81
  {
82
- "epoch": 0.2457002457002457,
83
- "grad_norm": 12.625,
84
- "learning_rate": 4.926108374384237e-05,
85
- "loss": 17.7765,
86
  "step": 50
87
  },
88
  {
89
- "epoch": 0.2702702702702703,
90
- "grad_norm": 24.875,
91
- "learning_rate": 5.41871921182266e-05,
92
- "loss": 16.5829,
 
 
 
 
 
 
 
 
93
  "step": 55
94
  },
95
  {
96
- "epoch": 0.29484029484029484,
97
- "grad_norm": 76.5,
98
- "learning_rate": 5.9113300492610844e-05,
99
- "loss": 13.7055,
100
  "step": 60
101
  },
102
  {
103
- "epoch": 0.3194103194103194,
104
- "grad_norm": 52.25,
105
- "learning_rate": 6.403940886699507e-05,
106
- "loss": 7.5296,
107
  "step": 65
108
  },
109
  {
110
- "epoch": 0.343980343980344,
111
- "grad_norm": 11.9375,
112
- "learning_rate": 6.896551724137931e-05,
113
- "loss": 2.7412,
114
  "step": 70
115
  },
116
  {
117
- "epoch": 0.36855036855036855,
118
- "grad_norm": 2.984375,
119
- "learning_rate": 7.389162561576355e-05,
120
- "loss": 2.1519,
121
  "step": 75
122
  },
123
  {
124
- "epoch": 0.3931203931203931,
125
- "grad_norm": 2.625,
126
- "learning_rate": 7.88177339901478e-05,
127
- "loss": 1.849,
128
  "step": 80
129
  },
130
  {
131
- "epoch": 0.4176904176904177,
132
- "grad_norm": 3.21875,
133
- "learning_rate": 8.374384236453202e-05,
134
- "loss": 1.6847,
135
  "step": 85
136
  },
137
  {
138
- "epoch": 0.44226044226044225,
139
- "grad_norm": 2.5625,
140
- "learning_rate": 8.866995073891627e-05,
141
- "loss": 1.528,
142
  "step": 90
143
  },
144
  {
145
- "epoch": 0.4668304668304668,
146
- "grad_norm": 5.65625,
147
- "learning_rate": 9.35960591133005e-05,
148
- "loss": 1.4458,
149
  "step": 95
150
  },
151
  {
152
- "epoch": 0.4914004914004914,
153
- "grad_norm": 1.96875,
154
- "learning_rate": 9.852216748768474e-05,
155
- "loss": 1.3912,
156
  "step": 100
157
  },
158
  {
159
- "epoch": 0.515970515970516,
160
- "grad_norm": 4.34375,
161
- "learning_rate": 0.00010344827586206898,
162
- "loss": 1.3435,
163
  "step": 105
164
  },
165
  {
166
- "epoch": 0.5405405405405406,
167
- "grad_norm": 1.484375,
168
- "learning_rate": 0.0001083743842364532,
169
- "loss": 1.2916,
 
 
 
 
 
 
 
 
170
  "step": 110
171
  },
172
  {
173
- "epoch": 0.5651105651105651,
174
- "grad_norm": 2.828125,
175
- "learning_rate": 0.00011330049261083743,
176
- "loss": 1.268,
177
  "step": 115
178
  },
179
  {
180
- "epoch": 0.5896805896805897,
181
- "grad_norm": 8.75,
182
- "learning_rate": 0.00011822660098522169,
183
- "loss": 1.2419,
184
  "step": 120
185
  },
186
  {
187
- "epoch": 0.6142506142506142,
188
- "grad_norm": 9.875,
189
- "learning_rate": 0.00012315270935960593,
190
- "loss": 1.1848,
191
  "step": 125
192
  },
193
  {
194
- "epoch": 0.6388206388206388,
195
- "grad_norm": 2.859375,
196
- "learning_rate": 0.00012807881773399014,
197
- "loss": 1.1821,
198
  "step": 130
199
  },
200
  {
201
- "epoch": 0.6633906633906634,
202
- "grad_norm": 2.203125,
203
- "learning_rate": 0.00013300492610837438,
204
- "loss": 1.1308,
205
  "step": 135
206
  },
207
  {
208
- "epoch": 0.687960687960688,
209
- "grad_norm": 3.453125,
210
- "learning_rate": 0.00013793103448275863,
211
- "loss": 1.1049,
212
  "step": 140
213
  },
214
  {
215
- "epoch": 0.7125307125307125,
216
- "grad_norm": 5.53125,
217
- "learning_rate": 0.00014285714285714287,
218
- "loss": 1.0904,
219
  "step": 145
220
  },
221
  {
222
- "epoch": 0.7371007371007371,
223
- "grad_norm": 36.75,
224
- "learning_rate": 0.0001477832512315271,
225
- "loss": 1.0884,
226
  "step": 150
227
  },
228
  {
229
- "epoch": 0.7616707616707616,
230
- "grad_norm": 2.1875,
231
- "learning_rate": 0.00015270935960591132,
232
- "loss": 1.0498,
233
  "step": 155
234
  },
235
  {
236
- "epoch": 0.7862407862407862,
237
- "grad_norm": 3.734375,
238
- "learning_rate": 0.0001576354679802956,
239
- "loss": 1.0656,
240
  "step": 160
241
  },
242
  {
243
- "epoch": 0.8108108108108109,
244
- "grad_norm": 2.71875,
245
- "learning_rate": 0.0001625615763546798,
246
- "loss": 1.0436,
247
  "step": 165
248
  },
249
  {
250
- "epoch": 0.8353808353808354,
251
- "grad_norm": 3.03125,
252
- "learning_rate": 0.00016748768472906405,
253
- "loss": 1.0507,
 
 
 
 
 
 
 
 
254
  "step": 170
255
  },
256
  {
257
- "epoch": 0.85995085995086,
258
- "grad_norm": 2.109375,
259
- "learning_rate": 0.00017241379310344826,
260
- "loss": 1.0593,
261
  "step": 175
262
  },
263
  {
264
- "epoch": 0.8845208845208845,
265
- "grad_norm": 28.875,
266
- "learning_rate": 0.00017733990147783253,
267
- "loss": 1.0432,
268
  "step": 180
269
  },
270
  {
271
- "epoch": 0.9090909090909091,
272
- "grad_norm": 1.4609375,
273
- "learning_rate": 0.00018226600985221675,
274
- "loss": 1.0558,
275
  "step": 185
276
  },
277
  {
278
- "epoch": 0.9336609336609336,
279
- "grad_norm": 3.796875,
280
- "learning_rate": 0.000187192118226601,
281
- "loss": 1.0305,
282
  "step": 190
283
  },
284
  {
285
- "epoch": 0.9582309582309583,
286
- "grad_norm": 1.9140625,
287
- "learning_rate": 0.00019211822660098523,
288
- "loss": 1.0315,
289
  "step": 195
290
  },
291
  {
292
- "epoch": 0.9828009828009828,
293
- "grad_norm": 3.28125,
294
- "learning_rate": 0.00019704433497536947,
295
- "loss": 1.0031,
296
  "step": 200
297
  },
298
  {
299
- "epoch": 0.9975429975429976,
300
- "eval_loss": 2.529916286468506,
301
- "eval_runtime": 2.0529,
302
- "eval_samples_per_second": 4.871,
303
- "eval_steps_per_second": 0.974,
304
- "step": 203
305
- },
306
- {
307
- "epoch": 1.0073710073710074,
308
- "grad_norm": 9.4375,
309
- "learning_rate": 0.00019999940863962815,
310
- "loss": 1.0212,
311
  "step": 205
312
  },
313
  {
314
- "epoch": 1.031941031941032,
315
- "grad_norm": 1.421875,
316
- "learning_rate": 0.00019999275591576766,
317
- "loss": 0.9729,
318
  "step": 210
319
  },
320
  {
321
- "epoch": 1.0565110565110565,
322
- "grad_norm": 5.875,
323
- "learning_rate": 0.00019997871176098827,
324
- "loss": 0.9684,
325
  "step": 215
326
  },
327
  {
328
- "epoch": 1.0810810810810811,
329
- "grad_norm": 3.328125,
330
- "learning_rate": 0.00019995727721342914,
331
- "loss": 0.9533,
332
  "step": 220
333
  },
334
  {
335
- "epoch": 1.1056511056511056,
336
- "grad_norm": 1.1953125,
337
- "learning_rate": 0.00019992845385752485,
338
- "loss": 0.9418,
 
 
 
 
 
 
 
 
339
  "step": 225
340
  },
341
  {
342
- "epoch": 1.1302211302211302,
343
- "grad_norm": 2.3125,
344
- "learning_rate": 0.00019989224382388813,
345
- "loss": 0.9365,
346
  "step": 230
347
  },
348
  {
349
- "epoch": 1.154791154791155,
350
- "grad_norm": 27.75,
351
- "learning_rate": 0.00019984864978915253,
352
- "loss": 0.9535,
353
  "step": 235
354
  },
355
  {
356
- "epoch": 1.1793611793611793,
357
- "grad_norm": 3.03125,
358
- "learning_rate": 0.00019979767497577445,
359
- "loss": 0.9555,
360
  "step": 240
361
  },
362
  {
363
- "epoch": 1.203931203931204,
364
- "grad_norm": 3.34375,
365
- "learning_rate": 0.000199739323151795,
366
- "loss": 0.9599,
367
  "step": 245
368
  },
369
  {
370
- "epoch": 1.2285012285012284,
371
- "grad_norm": 1.8671875,
372
- "learning_rate": 0.00019967359863056134,
373
- "loss": 0.9268,
374
  "step": 250
375
  },
376
  {
377
- "epoch": 1.253071253071253,
378
- "grad_norm": 1.96875,
379
- "learning_rate": 0.00019960050627040806,
380
- "loss": 0.9183,
381
  "step": 255
382
  },
383
  {
384
- "epoch": 1.2776412776412776,
385
- "grad_norm": 8.125,
386
- "learning_rate": 0.0001995200514742978,
387
- "loss": 0.9041,
388
  "step": 260
389
  },
390
  {
391
- "epoch": 1.3022113022113022,
392
- "grad_norm": 0.8046875,
393
- "learning_rate": 0.0001994322401894221,
394
- "loss": 0.915,
395
  "step": 265
396
  },
397
  {
398
- "epoch": 1.3267813267813269,
399
- "grad_norm": 0.96484375,
400
- "learning_rate": 0.00019933707890676158,
401
- "loss": 0.9009,
402
  "step": 270
403
  },
404
  {
405
- "epoch": 1.3513513513513513,
406
- "grad_norm": 3.515625,
407
- "learning_rate": 0.00019923457466060636,
408
- "loss": 0.918,
409
  "step": 275
410
  },
411
  {
412
- "epoch": 1.375921375921376,
413
- "grad_norm": 1.1640625,
414
- "learning_rate": 0.00019912473502803582,
415
- "loss": 0.9329,
 
 
 
 
 
 
 
 
416
  "step": 280
417
  },
418
  {
419
- "epoch": 1.4004914004914004,
420
- "grad_norm": 0.859375,
421
- "learning_rate": 0.0001990075681283587,
422
- "loss": 0.9269,
423
  "step": 285
424
  },
425
  {
426
- "epoch": 1.425061425061425,
427
- "grad_norm": 4.875,
428
- "learning_rate": 0.00019888308262251285,
429
- "loss": 0.9245,
430
  "step": 290
431
  },
432
  {
433
- "epoch": 1.4496314496314495,
434
- "grad_norm": 1.3203125,
435
- "learning_rate": 0.00019875128771242506,
436
- "loss": 0.9252,
437
  "step": 295
438
  },
439
  {
440
- "epoch": 1.4742014742014742,
441
- "grad_norm": 2.203125,
442
- "learning_rate": 0.00019861219314033077,
443
- "loss": 0.9055,
444
  "step": 300
445
  },
446
  {
447
- "epoch": 1.4987714987714988,
448
- "grad_norm": 2.640625,
449
- "learning_rate": 0.000198465809188054,
450
- "loss": 0.9095,
451
  "step": 305
452
  },
453
  {
454
- "epoch": 1.5233415233415233,
455
- "grad_norm": 1.7421875,
456
- "learning_rate": 0.0001983121466762474,
457
- "loss": 0.9061,
458
  "step": 310
459
  },
460
  {
461
- "epoch": 1.547911547911548,
462
- "grad_norm": 3.671875,
463
- "learning_rate": 0.00019815121696359212,
464
- "loss": 0.8923,
465
  "step": 315
466
  },
467
  {
468
- "epoch": 1.5724815724815726,
469
- "grad_norm": 2.875,
470
- "learning_rate": 0.00019798303194595846,
471
- "loss": 0.9021,
472
  "step": 320
473
  },
474
  {
475
- "epoch": 1.597051597051597,
476
- "grad_norm": 1.3671875,
477
- "learning_rate": 0.00019780760405552645,
478
- "loss": 0.8921,
479
  "step": 325
480
  },
481
  {
482
- "epoch": 1.6216216216216215,
483
- "grad_norm": 1.09375,
484
- "learning_rate": 0.00019762494625986677,
485
- "loss": 0.8885,
 
 
 
 
 
 
 
 
486
  "step": 330
487
  },
488
  {
489
- "epoch": 1.6461916461916462,
490
- "grad_norm": 2.671875,
491
- "learning_rate": 0.00019743507206098233,
492
- "loss": 0.8895,
493
  "step": 335
494
  },
495
  {
496
- "epoch": 1.6707616707616708,
497
- "grad_norm": 0.7265625,
498
- "learning_rate": 0.00019723799549431007,
499
- "loss": 0.8976,
500
  "step": 340
501
  },
502
  {
503
- "epoch": 1.6953316953316953,
504
- "grad_norm": 1.109375,
505
- "learning_rate": 0.00019703373112768365,
506
- "loss": 0.8836,
507
  "step": 345
508
  },
509
  {
510
- "epoch": 1.71990171990172,
511
- "grad_norm": 1.0234375,
512
- "learning_rate": 0.00019682229406025635,
513
- "loss": 0.8801,
514
  "step": 350
515
  },
516
  {
517
- "epoch": 1.7444717444717446,
518
- "grad_norm": 1.6875,
519
- "learning_rate": 0.00019660369992138517,
520
- "loss": 0.9048,
521
  "step": 355
522
  },
523
  {
524
- "epoch": 1.769041769041769,
525
- "grad_norm": 1.4921875,
526
- "learning_rate": 0.0001963779648694754,
527
- "loss": 0.8906,
528
  "step": 360
529
  },
530
  {
531
- "epoch": 1.7936117936117935,
532
- "grad_norm": 1.3203125,
533
- "learning_rate": 0.00019614510559078625,
534
- "loss": 0.8805,
535
  "step": 365
536
  },
537
  {
538
- "epoch": 1.8181818181818183,
539
- "grad_norm": 0.80859375,
540
- "learning_rate": 0.00019590513929819734,
541
- "loss": 0.8839,
542
  "step": 370
543
  },
544
  {
545
- "epoch": 1.8427518427518428,
546
- "grad_norm": 1.2109375,
547
- "learning_rate": 0.0001956580837299364,
548
- "loss": 0.8715,
549
  "step": 375
550
  },
551
  {
552
- "epoch": 1.8673218673218672,
553
- "grad_norm": 1.0078125,
554
- "learning_rate": 0.000195403957148268,
555
- "loss": 0.8952,
556
  "step": 380
557
  },
558
  {
559
- "epoch": 1.8918918918918919,
560
- "grad_norm": 0.75390625,
561
- "learning_rate": 0.0001951427783381437,
562
- "loss": 0.8585,
 
 
 
 
 
 
 
 
563
  "step": 385
564
  },
565
  {
566
- "epoch": 1.9164619164619165,
567
- "grad_norm": 0.953125,
568
- "learning_rate": 0.0001948745666058134,
569
- "loss": 0.8868,
570
  "step": 390
571
  },
572
  {
573
- "epoch": 1.941031941031941,
574
- "grad_norm": 2.734375,
575
- "learning_rate": 0.00019459934177739813,
576
- "loss": 0.8778,
577
  "step": 395
578
  },
579
  {
580
- "epoch": 1.9656019656019657,
581
- "grad_norm": 1.8359375,
582
- "learning_rate": 0.00019431712419742484,
583
- "loss": 0.8902,
584
  "step": 400
585
  },
586
  {
587
- "epoch": 1.9901719901719903,
588
- "grad_norm": 3.078125,
589
- "learning_rate": 0.00019402793472732217,
590
- "loss": 0.8685,
591
  "step": 405
592
  },
593
  {
594
- "epoch": 2.0,
595
- "eval_loss": 2.4260478019714355,
596
- "eval_runtime": 2.0431,
597
- "eval_samples_per_second": 4.895,
598
- "eval_steps_per_second": 0.979,
599
- "step": 407
600
- },
601
- {
602
- "epoch": 2.0147420147420148,
603
- "grad_norm": 1.359375,
604
- "learning_rate": 0.00019373179474387858,
605
- "loss": 0.8187,
606
  "step": 410
607
  },
608
  {
609
- "epoch": 2.039312039312039,
610
- "grad_norm": 0.9140625,
611
- "learning_rate": 0.0001934287261376622,
612
- "loss": 0.7808,
613
  "step": 415
614
  },
615
  {
616
- "epoch": 2.063882063882064,
617
- "grad_norm": 0.73828125,
618
- "learning_rate": 0.00019311875131140246,
619
- "loss": 0.7746,
620
  "step": 420
621
  },
622
  {
623
- "epoch": 2.0884520884520885,
624
- "grad_norm": 0.71875,
625
- "learning_rate": 0.00019280189317833445,
626
- "loss": 0.7761,
627
  "step": 425
628
  },
629
  {
630
- "epoch": 2.113022113022113,
631
- "grad_norm": 1.5859375,
632
- "learning_rate": 0.00019247817516050483,
633
- "loss": 0.7781,
634
  "step": 430
635
  },
636
  {
637
- "epoch": 2.1375921375921374,
638
- "grad_norm": 0.93359375,
639
- "learning_rate": 0.00019214762118704076,
640
- "loss": 0.7648,
641
  "step": 435
642
  },
643
  {
644
- "epoch": 2.1621621621621623,
645
- "grad_norm": 0.828125,
646
- "learning_rate": 0.0001918102556923809,
647
- "loss": 0.7926,
 
 
 
 
 
 
 
 
648
  "step": 440
649
  },
650
  {
651
- "epoch": 2.1867321867321867,
652
- "grad_norm": 2.0625,
653
- "learning_rate": 0.0001914661036144692,
654
- "loss": 0.7821,
655
  "step": 445
656
  },
657
  {
658
- "epoch": 2.211302211302211,
659
- "grad_norm": 1.0234375,
660
- "learning_rate": 0.00019111519039291167,
661
- "loss": 0.7807,
662
  "step": 450
663
  },
664
  {
665
- "epoch": 2.235872235872236,
666
- "grad_norm": 1.484375,
667
- "learning_rate": 0.00019075754196709572,
668
- "loss": 0.7994,
669
  "step": 455
670
  },
671
  {
672
- "epoch": 2.2604422604422605,
673
- "grad_norm": 1.0625,
674
- "learning_rate": 0.0001903931847742728,
675
- "loss": 0.7777,
676
  "step": 460
677
  },
678
  {
679
- "epoch": 2.285012285012285,
680
- "grad_norm": 1.8203125,
681
- "learning_rate": 0.00019002214574760423,
682
- "loss": 0.7781,
683
  "step": 465
684
  },
685
  {
686
- "epoch": 2.30958230958231,
687
- "grad_norm": 1.484375,
688
- "learning_rate": 0.0001896444523141701,
689
- "loss": 0.7977,
690
  "step": 470
691
  },
692
  {
693
- "epoch": 2.3341523341523343,
694
- "grad_norm": 0.73046875,
695
- "learning_rate": 0.00018926013239294216,
696
- "loss": 0.7758,
697
  "step": 475
698
  },
699
  {
700
- "epoch": 2.3587223587223587,
701
- "grad_norm": 1.0234375,
702
- "learning_rate": 0.00018886921439271984,
703
- "loss": 0.8019,
704
  "step": 480
705
  },
706
  {
707
- "epoch": 2.383292383292383,
708
- "grad_norm": 0.91015625,
709
- "learning_rate": 0.00018847172721003043,
710
- "loss": 0.7829,
711
  "step": 485
712
  },
713
  {
714
- "epoch": 2.407862407862408,
715
- "grad_norm": 0.984375,
716
- "learning_rate": 0.00018806770022699278,
717
- "loss": 0.7759,
718
  "step": 490
719
  },
720
  {
721
- "epoch": 2.4324324324324325,
722
- "grad_norm": 0.8828125,
723
- "learning_rate": 0.0001876571633091458,
724
- "loss": 0.776,
 
 
 
 
 
 
 
 
725
  "step": 495
726
  },
727
  {
728
- "epoch": 2.457002457002457,
729
- "grad_norm": 1.1875,
730
- "learning_rate": 0.00018724014680324057,
731
- "loss": 0.7885,
732
  "step": 500
733
  },
734
  {
735
- "epoch": 2.4815724815724813,
736
- "grad_norm": 0.9375,
737
- "learning_rate": 0.00018681668153499697,
738
- "loss": 0.7929,
739
  "step": 505
740
  },
741
  {
742
- "epoch": 2.506142506142506,
743
- "grad_norm": 4.25,
744
- "learning_rate": 0.00018638679880682543,
745
- "loss": 0.7835,
746
  "step": 510
747
  },
748
  {
749
- "epoch": 2.5307125307125307,
750
- "grad_norm": 1.109375,
751
- "learning_rate": 0.00018595053039551274,
752
- "loss": 0.7981,
753
  "step": 515
754
  },
755
  {
756
- "epoch": 2.555282555282555,
757
- "grad_norm": 0.80078125,
758
- "learning_rate": 0.00018550790854987323,
759
- "loss": 0.7831,
760
  "step": 520
761
  },
762
  {
763
- "epoch": 2.57985257985258,
764
- "grad_norm": 0.92578125,
765
- "learning_rate": 0.00018505896598836508,
766
- "loss": 0.7863,
767
  "step": 525
768
  },
769
  {
770
- "epoch": 2.6044226044226044,
771
- "grad_norm": 0.84375,
772
- "learning_rate": 0.00018460373589667154,
773
- "loss": 0.7929,
774
  "step": 530
775
  },
776
  {
777
- "epoch": 2.628992628992629,
778
- "grad_norm": 1.1015625,
779
- "learning_rate": 0.00018414225192524806,
780
- "loss": 0.7722,
781
  "step": 535
782
  },
783
  {
784
- "epoch": 2.6535626535626538,
785
- "grad_norm": 1.390625,
786
- "learning_rate": 0.00018367454818683473,
787
- "loss": 0.7832,
788
  "step": 540
789
  },
790
  {
791
- "epoch": 2.678132678132678,
792
- "grad_norm": 1.2890625,
793
- "learning_rate": 0.00018320065925393468,
794
- "loss": 0.7746,
795
  "step": 545
796
  },
797
  {
798
- "epoch": 2.7027027027027026,
799
- "grad_norm": 1.59375,
800
- "learning_rate": 0.00018272062015625872,
801
- "loss": 0.7826,
802
  "step": 550
803
  },
804
  {
805
- "epoch": 2.7272727272727275,
806
- "grad_norm": 1.328125,
807
- "learning_rate": 0.0001822344663781356,
808
- "loss": 0.7946,
809
- "step": 555
810
- },
811
- {
812
- "epoch": 2.751842751842752,
813
- "grad_norm": 1.0234375,
814
- "learning_rate": 0.00018174223385588917,
815
- "loss": 0.7849,
816
- "step": 560
817
- },
818
- {
819
- "epoch": 2.7764127764127764,
820
- "grad_norm": 10.3125,
821
- "learning_rate": 0.00018124395897518224,
822
- "loss": 0.7783,
823
- "step": 565
824
- },
825
- {
826
- "epoch": 2.800982800982801,
827
- "grad_norm": 0.8515625,
828
- "learning_rate": 0.0001807396785683264,
829
- "loss": 0.8005,
830
- "step": 570
831
- },
832
- {
833
- "epoch": 2.8255528255528253,
834
- "grad_norm": 0.83984375,
835
- "learning_rate": 0.00018022942991156,
836
- "loss": 0.7981,
837
- "step": 575
838
- },
839
- {
840
- "epoch": 2.85012285012285,
841
- "grad_norm": 1.015625,
842
- "learning_rate": 0.00017971325072229226,
843
- "loss": 0.7887,
844
- "step": 580
845
- },
846
- {
847
- "epoch": 2.8746928746928746,
848
- "grad_norm": 0.8125,
849
- "learning_rate": 0.0001791911791563154,
850
- "loss": 0.8013,
851
- "step": 585
852
- },
853
- {
854
- "epoch": 2.899262899262899,
855
- "grad_norm": 1.2578125,
856
- "learning_rate": 0.00017866325380498416,
857
- "loss": 0.777,
858
- "step": 590
859
- },
860
- {
861
- "epoch": 2.923832923832924,
862
- "grad_norm": 1.03125,
863
- "learning_rate": 0.00017812951369236316,
864
- "loss": 0.7992,
865
- "step": 595
866
- },
867
- {
868
- "epoch": 2.9484029484029484,
869
- "grad_norm": 1.640625,
870
- "learning_rate": 0.00017758999827234212,
871
- "loss": 0.7856,
872
- "step": 600
873
- },
874
- {
875
- "epoch": 2.972972972972973,
876
- "grad_norm": 3.109375,
877
- "learning_rate": 0.00017704474742571969,
878
- "loss": 0.787,
879
- "step": 605
880
- },
881
- {
882
- "epoch": 2.9975429975429977,
883
- "grad_norm": 3.8125,
884
- "learning_rate": 0.00017649380145725517,
885
- "loss": 0.8,
886
- "step": 610
887
- },
888
- {
889
- "epoch": 2.9975429975429977,
890
- "eval_loss": 2.5051403045654297,
891
- "eval_runtime": 2.0509,
892
- "eval_samples_per_second": 4.876,
893
- "eval_steps_per_second": 0.975,
894
- "step": 610
895
- },
896
- {
897
- "epoch": 3.022113022113022,
898
- "grad_norm": 1.7890625,
899
- "learning_rate": 0.00017593720109268944,
900
- "loss": 0.6916,
901
- "step": 615
902
- },
903
- {
904
- "epoch": 3.0466830466830466,
905
- "grad_norm": 9.4375,
906
- "learning_rate": 0.00017537498747573443,
907
- "loss": 0.6614,
908
- "step": 620
909
- },
910
- {
911
- "epoch": 3.0712530712530715,
912
- "grad_norm": 0.96484375,
913
- "learning_rate": 0.00017480720216503183,
914
- "loss": 0.6639,
915
- "step": 625
916
- },
917
- {
918
- "epoch": 3.095823095823096,
919
- "grad_norm": 0.74609375,
920
- "learning_rate": 0.000174233887131081,
921
- "loss": 0.6579,
922
- "step": 630
923
- },
924
- {
925
- "epoch": 3.1203931203931203,
926
- "grad_norm": 1.015625,
927
- "learning_rate": 0.0001736550847531366,
928
- "loss": 0.6546,
929
- "step": 635
930
- },
931
- {
932
- "epoch": 3.1449631449631448,
933
- "grad_norm": 0.8203125,
934
- "learning_rate": 0.00017307083781607595,
935
- "loss": 0.6731,
936
- "step": 640
937
- },
938
- {
939
- "epoch": 3.1695331695331697,
940
- "grad_norm": 0.87890625,
941
- "learning_rate": 0.00017248118950723634,
942
- "loss": 0.6761,
943
- "step": 645
944
- },
945
- {
946
- "epoch": 3.194103194103194,
947
- "grad_norm": 0.98828125,
948
- "learning_rate": 0.00017188618341322254,
949
- "loss": 0.6761,
950
- "step": 650
951
- },
952
- {
953
- "epoch": 3.2186732186732185,
954
- "grad_norm": 1.7421875,
955
- "learning_rate": 0.00017128586351668524,
956
- "loss": 0.666,
957
- "step": 655
958
- },
959
- {
960
- "epoch": 3.2432432432432434,
961
- "grad_norm": 1.3515625,
962
- "learning_rate": 0.00017068027419306936,
963
- "loss": 0.6677,
964
- "step": 660
965
- },
966
- {
967
- "epoch": 3.267813267813268,
968
- "grad_norm": 0.9609375,
969
- "learning_rate": 0.00017006946020733425,
970
- "loss": 0.6663,
971
- "step": 665
972
- },
973
- {
974
- "epoch": 3.2923832923832923,
975
- "grad_norm": 0.77734375,
976
- "learning_rate": 0.00016945346671064452,
977
- "loss": 0.6762,
978
- "step": 670
979
- },
980
- {
981
- "epoch": 3.3169533169533167,
982
- "grad_norm": 0.7578125,
983
- "learning_rate": 0.00016883233923703248,
984
- "loss": 0.6842,
985
- "step": 675
986
- },
987
- {
988
- "epoch": 3.3415233415233416,
989
- "grad_norm": 1.359375,
990
- "learning_rate": 0.00016820612370003221,
991
- "loss": 0.6756,
992
- "step": 680
993
- },
994
- {
995
- "epoch": 3.366093366093366,
996
- "grad_norm": 1.015625,
997
- "learning_rate": 0.00016757486638928587,
998
- "loss": 0.6757,
999
- "step": 685
1000
- },
1001
- {
1002
- "epoch": 3.3906633906633905,
1003
- "grad_norm": 1.5390625,
1004
- "learning_rate": 0.00016693861396712168,
1005
- "loss": 0.6971,
1006
- "step": 690
1007
- },
1008
- {
1009
- "epoch": 3.4152334152334154,
1010
- "grad_norm": 1.125,
1011
- "learning_rate": 0.00016629741346510496,
1012
- "loss": 0.6837,
1013
- "step": 695
1014
- },
1015
- {
1016
- "epoch": 3.43980343980344,
1017
- "grad_norm": 2.6875,
1018
- "learning_rate": 0.00016565131228056133,
1019
- "loss": 0.6836,
1020
- "step": 700
1021
- },
1022
- {
1023
- "epoch": 3.4643734643734643,
1024
- "grad_norm": 0.77734375,
1025
- "learning_rate": 0.00016500035817307334,
1026
- "loss": 0.6719,
1027
- "step": 705
1028
- },
1029
- {
1030
- "epoch": 3.488943488943489,
1031
- "grad_norm": 0.75,
1032
- "learning_rate": 0.0001643445992609498,
1033
- "loss": 0.6722,
1034
- "step": 710
1035
- },
1036
- {
1037
- "epoch": 3.5135135135135136,
1038
- "grad_norm": 0.82421875,
1039
- "learning_rate": 0.00016368408401766916,
1040
- "loss": 0.6843,
1041
- "step": 715
1042
- },
1043
- {
1044
- "epoch": 3.538083538083538,
1045
- "grad_norm": 0.85546875,
1046
- "learning_rate": 0.0001630188612682963,
1047
- "loss": 0.6787,
1048
- "step": 720
1049
- },
1050
- {
1051
- "epoch": 3.562653562653563,
1052
- "grad_norm": 0.71875,
1053
- "learning_rate": 0.00016234898018587337,
1054
- "loss": 0.6639,
1055
- "step": 725
1056
- },
1057
- {
1058
- "epoch": 3.5872235872235874,
1059
- "grad_norm": 0.96484375,
1060
- "learning_rate": 0.00016167449028778484,
1061
- "loss": 0.6951,
1062
- "step": 730
1063
- },
1064
- {
1065
- "epoch": 3.611793611793612,
1066
- "grad_norm": 0.80859375,
1067
- "learning_rate": 0.0001609954414320973,
1068
- "loss": 0.6881,
1069
- "step": 735
1070
- },
1071
- {
1072
- "epoch": 3.6363636363636362,
1073
- "grad_norm": 0.703125,
1074
- "learning_rate": 0.0001603118838138741,
1075
- "loss": 0.6761,
1076
- "step": 740
1077
- },
1078
- {
1079
- "epoch": 3.6609336609336607,
1080
- "grad_norm": 0.9609375,
1081
- "learning_rate": 0.00015962386796146462,
1082
- "loss": 0.69,
1083
- "step": 745
1084
- },
1085
- {
1086
- "epoch": 3.6855036855036856,
1087
- "grad_norm": 0.79296875,
1088
- "learning_rate": 0.00015893144473276953,
1089
- "loss": 0.691,
1090
- "step": 750
1091
- },
1092
- {
1093
- "epoch": 3.71007371007371,
1094
- "grad_norm": 1.03125,
1095
- "learning_rate": 0.00015823466531148124,
1096
- "loss": 0.6843,
1097
- "step": 755
1098
- },
1099
- {
1100
- "epoch": 3.7346437346437344,
1101
- "grad_norm": 0.765625,
1102
- "learning_rate": 0.00015753358120330042,
1103
- "loss": 0.7094,
1104
- "step": 760
1105
- },
1106
- {
1107
- "epoch": 3.7592137592137593,
1108
- "grad_norm": 1.0078125,
1109
- "learning_rate": 0.00015682824423212877,
1110
- "loss": 0.6892,
1111
- "step": 765
1112
- },
1113
- {
1114
- "epoch": 3.7837837837837838,
1115
- "grad_norm": 0.9296875,
1116
- "learning_rate": 0.00015611870653623825,
1117
- "loss": 0.6929,
1118
- "step": 770
1119
- },
1120
- {
1121
- "epoch": 3.808353808353808,
1122
- "grad_norm": 1.53125,
1123
- "learning_rate": 0.00015540502056441688,
1124
- "loss": 0.7022,
1125
- "step": 775
1126
- },
1127
- {
1128
- "epoch": 3.832923832923833,
1129
- "grad_norm": 0.91015625,
1130
- "learning_rate": 0.00015468723907209193,
1131
- "loss": 0.703,
1132
- "step": 780
1133
- },
1134
- {
1135
- "epoch": 3.8574938574938575,
1136
- "grad_norm": 0.80078125,
1137
- "learning_rate": 0.00015396541511743012,
1138
- "loss": 0.7027,
1139
- "step": 785
1140
- },
1141
- {
1142
- "epoch": 3.882063882063882,
1143
- "grad_norm": 0.98046875,
1144
- "learning_rate": 0.00015323960205741561,
1145
- "loss": 0.6829,
1146
- "step": 790
1147
- },
1148
- {
1149
- "epoch": 3.906633906633907,
1150
- "grad_norm": 0.921875,
1151
- "learning_rate": 0.00015250985354390596,
1152
- "loss": 0.6945,
1153
- "step": 795
1154
- },
1155
- {
1156
- "epoch": 3.9312039312039313,
1157
- "grad_norm": 0.98046875,
1158
- "learning_rate": 0.0001517762235196661,
1159
- "loss": 0.6966,
1160
- "step": 800
1161
- },
1162
- {
1163
- "epoch": 3.9557739557739557,
1164
- "grad_norm": 0.7890625,
1165
- "learning_rate": 0.00015103876621438086,
1166
- "loss": 0.6947,
1167
- "step": 805
1168
- },
1169
- {
1170
- "epoch": 3.98034398034398,
1171
- "grad_norm": 0.89453125,
1172
- "learning_rate": 0.00015029753614064645,
1173
- "loss": 0.6938,
1174
- "step": 810
1175
- },
1176
- {
1177
- "epoch": 4.0,
1178
- "eval_loss": 2.6558120250701904,
1179
- "eval_runtime": 2.0446,
1180
- "eval_samples_per_second": 4.891,
1181
- "eval_steps_per_second": 0.978,
1182
- "step": 814
1183
- },
1184
- {
1185
- "epoch": 4.004914004914005,
1186
- "grad_norm": 1.5703125,
1187
- "learning_rate": 0.00014955258808994096,
1188
- "loss": 0.6676,
1189
- "step": 815
1190
- },
1191
- {
1192
- "epoch": 4.0294840294840295,
1193
- "grad_norm": 0.92578125,
1194
- "learning_rate": 0.00014880397712857386,
1195
- "loss": 0.5659,
1196
- "step": 820
1197
- },
1198
- {
1199
- "epoch": 4.054054054054054,
1200
- "grad_norm": 0.7890625,
1201
- "learning_rate": 0.00014805175859361594,
1202
- "loss": 0.5525,
1203
- "step": 825
1204
- },
1205
- {
1206
- "epoch": 4.078624078624078,
1207
- "grad_norm": 1.328125,
1208
- "learning_rate": 0.00014729598808880861,
1209
- "loss": 0.5546,
1210
- "step": 830
1211
- },
1212
- {
1213
- "epoch": 4.103194103194103,
1214
- "grad_norm": 0.96875,
1215
- "learning_rate": 0.00014653672148045357,
1216
- "loss": 0.5665,
1217
- "step": 835
1218
- },
1219
- {
1220
- "epoch": 4.127764127764128,
1221
- "grad_norm": 1.0078125,
1222
- "learning_rate": 0.00014577401489328335,
1223
- "loss": 0.565,
1224
- "step": 840
1225
- },
1226
- {
1227
- "epoch": 4.152334152334152,
1228
- "grad_norm": 0.7734375,
1229
- "learning_rate": 0.0001450079247063127,
1230
- "loss": 0.5764,
1231
- "step": 845
1232
- },
1233
- {
1234
- "epoch": 4.176904176904177,
1235
- "grad_norm": 0.9375,
1236
- "learning_rate": 0.00014423850754867075,
1237
- "loss": 0.5565,
1238
- "step": 850
1239
- },
1240
- {
1241
- "epoch": 4.201474201474202,
1242
- "grad_norm": 0.921875,
1243
- "learning_rate": 0.0001434658202954153,
1244
- "loss": 0.5572,
1245
- "step": 855
1246
- },
1247
- {
1248
- "epoch": 4.226044226044226,
1249
- "grad_norm": 2.0,
1250
- "learning_rate": 0.00014268992006332846,
1251
- "loss": 0.5719,
1252
- "step": 860
1253
- },
1254
- {
1255
- "epoch": 4.250614250614251,
1256
- "grad_norm": 0.98828125,
1257
- "learning_rate": 0.0001419108642066947,
1258
- "loss": 0.5644,
1259
- "step": 865
1260
- },
1261
- {
1262
- "epoch": 4.275184275184275,
1263
- "grad_norm": 0.88671875,
1264
- "learning_rate": 0.00014112871031306119,
1265
- "loss": 0.5785,
1266
- "step": 870
1267
- },
1268
- {
1269
- "epoch": 4.2997542997543,
1270
- "grad_norm": 1.734375,
1271
- "learning_rate": 0.00014034351619898088,
1272
- "loss": 0.5825,
1273
- "step": 875
1274
- },
1275
- {
1276
- "epoch": 4.324324324324325,
1277
- "grad_norm": 1.1171875,
1278
- "learning_rate": 0.00013955533990573886,
1279
- "loss": 0.5752,
1280
- "step": 880
1281
- },
1282
- {
1283
- "epoch": 4.348894348894349,
1284
- "grad_norm": 1.1484375,
1285
- "learning_rate": 0.00013876423969506194,
1286
- "loss": 0.5863,
1287
- "step": 885
1288
- },
1289
- {
1290
- "epoch": 4.3734643734643734,
1291
- "grad_norm": 1.7421875,
1292
- "learning_rate": 0.00013797027404481184,
1293
- "loss": 0.5826,
1294
- "step": 890
1295
- },
1296
- {
1297
- "epoch": 4.398034398034398,
1298
- "grad_norm": 0.99609375,
1299
- "learning_rate": 0.0001371735016446627,
1300
- "loss": 0.576,
1301
- "step": 895
1302
- },
1303
- {
1304
- "epoch": 4.422604422604422,
1305
- "grad_norm": 0.875,
1306
- "learning_rate": 0.00013637398139176255,
1307
- "loss": 0.577,
1308
- "step": 900
1309
- },
1310
- {
1311
- "epoch": 4.447174447174447,
1312
- "grad_norm": 0.9375,
1313
- "learning_rate": 0.00013557177238637986,
1314
- "loss": 0.5832,
1315
- "step": 905
1316
- },
1317
- {
1318
- "epoch": 4.471744471744472,
1319
- "grad_norm": 0.9375,
1320
- "learning_rate": 0.00013476693392753476,
1321
- "loss": 0.5856,
1322
- "step": 910
1323
- },
1324
- {
1325
- "epoch": 4.496314496314496,
1326
- "grad_norm": 1.1328125,
1327
- "learning_rate": 0.00013395952550861572,
1328
- "loss": 0.592,
1329
- "step": 915
1330
- },
1331
- {
1332
- "epoch": 4.520884520884521,
1333
- "grad_norm": 0.97265625,
1334
- "learning_rate": 0.00013314960681298175,
1335
- "loss": 0.5861,
1336
- "step": 920
1337
- },
1338
- {
1339
- "epoch": 4.545454545454545,
1340
- "grad_norm": 0.97265625,
1341
- "learning_rate": 0.0001323372377095507,
1342
- "loss": 0.5814,
1343
- "step": 925
1344
- },
1345
- {
1346
- "epoch": 4.57002457002457,
1347
- "grad_norm": 0.91796875,
1348
- "learning_rate": 0.0001315224782483737,
1349
- "loss": 0.5847,
1350
- "step": 930
1351
- },
1352
- {
1353
- "epoch": 4.594594594594595,
1354
- "grad_norm": 0.8203125,
1355
- "learning_rate": 0.00013070538865619642,
1356
- "loss": 0.5773,
1357
- "step": 935
1358
- },
1359
- {
1360
- "epoch": 4.61916461916462,
1361
- "grad_norm": 0.859375,
1362
- "learning_rate": 0.00012988602933200689,
1363
- "loss": 0.5723,
1364
- "step": 940
1365
- },
1366
- {
1367
- "epoch": 4.643734643734644,
1368
- "grad_norm": 1.1171875,
1369
- "learning_rate": 0.0001290644608425711,
1370
- "loss": 0.5918,
1371
- "step": 945
1372
- },
1373
- {
1374
- "epoch": 4.6683046683046685,
1375
- "grad_norm": 0.90234375,
1376
- "learning_rate": 0.0001282407439179557,
1377
- "loss": 0.5932,
1378
- "step": 950
1379
- },
1380
- {
1381
- "epoch": 4.6928746928746925,
1382
- "grad_norm": 0.8671875,
1383
- "learning_rate": 0.00012741493944703905,
1384
- "loss": 0.5868,
1385
- "step": 955
1386
- },
1387
- {
1388
- "epoch": 4.717444717444717,
1389
- "grad_norm": 0.83203125,
1390
- "learning_rate": 0.0001265871084730101,
1391
- "loss": 0.5944,
1392
- "step": 960
1393
- },
1394
- {
1395
- "epoch": 4.742014742014742,
1396
- "grad_norm": 0.9765625,
1397
- "learning_rate": 0.00012575731218885625,
1398
- "loss": 0.5893,
1399
- "step": 965
1400
- },
1401
- {
1402
- "epoch": 4.766584766584766,
1403
- "grad_norm": 1.140625,
1404
- "learning_rate": 0.00012492561193284008,
1405
- "loss": 0.5791,
1406
- "step": 970
1407
- },
1408
- {
1409
- "epoch": 4.791154791154791,
1410
- "grad_norm": 1.1484375,
1411
- "learning_rate": 0.00012409206918396503,
1412
- "loss": 0.5876,
1413
- "step": 975
1414
- },
1415
- {
1416
- "epoch": 4.815724815724816,
1417
- "grad_norm": 0.7890625,
1418
- "learning_rate": 0.00012325674555743106,
1419
- "loss": 0.5871,
1420
- "step": 980
1421
- },
1422
- {
1423
- "epoch": 4.84029484029484,
1424
- "grad_norm": 0.8046875,
1425
- "learning_rate": 0.0001224197028000799,
1426
- "loss": 0.5987,
1427
- "step": 985
1428
- },
1429
- {
1430
- "epoch": 4.864864864864865,
1431
- "grad_norm": 0.92578125,
1432
- "learning_rate": 0.000121581002785831,
1433
- "loss": 0.5857,
1434
- "step": 990
1435
- },
1436
- {
1437
- "epoch": 4.88943488943489,
1438
- "grad_norm": 0.80078125,
1439
- "learning_rate": 0.00012074070751110751,
1440
- "loss": 0.5932,
1441
- "step": 995
1442
- },
1443
- {
1444
- "epoch": 4.914004914004914,
1445
- "grad_norm": 1.0625,
1446
- "learning_rate": 0.00011989887909025388,
1447
- "loss": 0.5891,
1448
- "step": 1000
1449
- },
1450
- {
1451
- "epoch": 4.938574938574939,
1452
- "grad_norm": 0.796875,
1453
- "learning_rate": 0.00011905557975094406,
1454
- "loss": 0.5881,
1455
- "step": 1005
1456
- },
1457
- {
1458
- "epoch": 4.963144963144963,
1459
- "grad_norm": 0.87890625,
1460
- "learning_rate": 0.00011821087182958186,
1461
- "loss": 0.5845,
1462
- "step": 1010
1463
- },
1464
- {
1465
- "epoch": 4.987714987714988,
1466
- "grad_norm": 0.828125,
1467
- "learning_rate": 0.00011736481776669306,
1468
- "loss": 0.5865,
1469
- "step": 1015
1470
- },
1471
- {
1472
- "epoch": 4.997542997542998,
1473
- "eval_loss": 2.9162657260894775,
1474
- "eval_runtime": 2.0535,
1475
- "eval_samples_per_second": 4.87,
1476
- "eval_steps_per_second": 0.974,
1477
- "step": 1017
1478
- },
1479
- {
1480
- "epoch": 5.012285012285012,
1481
- "grad_norm": 0.75390625,
1482
- "learning_rate": 0.0001165174801023096,
1483
- "loss": 0.5388,
1484
- "step": 1020
1485
- },
1486
- {
1487
- "epoch": 5.036855036855036,
1488
- "grad_norm": 0.953125,
1489
- "learning_rate": 0.00011566892147134705,
1490
- "loss": 0.4596,
1491
- "step": 1025
1492
- },
1493
- {
1494
- "epoch": 5.061425061425061,
1495
- "grad_norm": 0.7890625,
1496
- "learning_rate": 0.00011481920459897417,
1497
- "loss": 0.4681,
1498
- "step": 1030
1499
- },
1500
- {
1501
- "epoch": 5.085995085995086,
1502
- "grad_norm": 0.8828125,
1503
- "learning_rate": 0.00011396839229597674,
1504
- "loss": 0.4716,
1505
- "step": 1035
1506
- },
1507
- {
1508
- "epoch": 5.11056511056511,
1509
- "grad_norm": 0.80078125,
1510
- "learning_rate": 0.00011311654745411425,
1511
- "loss": 0.4667,
1512
- "step": 1040
1513
- },
1514
- {
1515
- "epoch": 5.135135135135135,
1516
- "grad_norm": 0.90625,
1517
- "learning_rate": 0.00011226373304147123,
1518
- "loss": 0.465,
1519
- "step": 1045
1520
- },
1521
- {
1522
- "epoch": 5.15970515970516,
1523
- "grad_norm": 0.828125,
1524
- "learning_rate": 0.00011141001209780249,
1525
- "loss": 0.4751,
1526
- "step": 1050
1527
- },
1528
- {
1529
- "epoch": 5.184275184275184,
1530
- "grad_norm": 1.0390625,
1531
- "learning_rate": 0.00011055544772987335,
1532
- "loss": 0.4622,
1533
- "step": 1055
1534
- },
1535
- {
1536
- "epoch": 5.208845208845209,
1537
- "grad_norm": 1.234375,
1538
- "learning_rate": 0.0001097001031067947,
1539
- "loss": 0.4839,
1540
- "step": 1060
1541
- },
1542
- {
1543
- "epoch": 5.233415233415234,
1544
- "grad_norm": 0.91015625,
1545
- "learning_rate": 0.00010884404145535372,
1546
- "loss": 0.4679,
1547
- "step": 1065
1548
- },
1549
- {
1550
- "epoch": 5.257985257985258,
1551
- "grad_norm": 0.88671875,
1552
- "learning_rate": 0.00010798732605534006,
1553
- "loss": 0.4764,
1554
- "step": 1070
1555
- },
1556
- {
1557
- "epoch": 5.282555282555283,
1558
- "grad_norm": 0.90234375,
1559
- "learning_rate": 0.00010713002023486816,
1560
- "loss": 0.4834,
1561
- "step": 1075
1562
- },
1563
- {
1564
- "epoch": 5.3071253071253075,
1565
- "grad_norm": 0.98046875,
1566
- "learning_rate": 0.00010627218736569624,
1567
- "loss": 0.4853,
1568
- "step": 1080
1569
- },
1570
- {
1571
- "epoch": 5.3316953316953315,
1572
- "grad_norm": 0.875,
1573
- "learning_rate": 0.00010541389085854176,
1574
- "loss": 0.4823,
1575
- "step": 1085
1576
- },
1577
- {
1578
- "epoch": 5.356265356265356,
1579
- "grad_norm": 1.03125,
1580
- "learning_rate": 0.00010455519415839415,
1581
- "loss": 0.4838,
1582
- "step": 1090
1583
- },
1584
- {
1585
- "epoch": 5.38083538083538,
1586
- "grad_norm": 0.92578125,
1587
- "learning_rate": 0.00010369616073982491,
1588
- "loss": 0.4881,
1589
- "step": 1095
1590
- },
1591
- {
1592
- "epoch": 5.405405405405405,
1593
- "grad_norm": 0.88671875,
1594
- "learning_rate": 0.00010283685410229571,
1595
- "loss": 0.485,
1596
- "step": 1100
1597
- },
1598
- {
1599
- "epoch": 5.42997542997543,
1600
- "grad_norm": 1.0390625,
1601
- "learning_rate": 0.00010197733776546447,
1602
- "loss": 0.4854,
1603
- "step": 1105
1604
- },
1605
- {
1606
- "epoch": 5.454545454545454,
1607
- "grad_norm": 0.9609375,
1608
- "learning_rate": 0.00010111767526449004,
1609
- "loss": 0.4851,
1610
- "step": 1110
1611
- },
1612
- {
1613
- "epoch": 5.479115479115479,
1614
- "grad_norm": 1.265625,
1615
- "learning_rate": 0.00010025793014533558,
1616
- "loss": 0.4889,
1617
- "step": 1115
1618
- },
1619
- {
1620
- "epoch": 5.503685503685504,
1621
- "grad_norm": 0.87890625,
1622
- "learning_rate": 9.939816596007146e-05,
1623
- "loss": 0.4772,
1624
- "step": 1120
1625
- },
1626
- {
1627
- "epoch": 5.528255528255528,
1628
- "grad_norm": 1.1640625,
1629
- "learning_rate": 9.853844626217737e-05,
1630
- "loss": 0.4857,
1631
- "step": 1125
1632
- },
1633
- {
1634
- "epoch": 5.552825552825553,
1635
- "grad_norm": 1.0703125,
1636
- "learning_rate": 9.767883460184443e-05,
1637
- "loss": 0.4869,
1638
- "step": 1130
1639
- },
1640
- {
1641
- "epoch": 5.577395577395578,
1642
- "grad_norm": 0.88671875,
1643
- "learning_rate": 9.681939452127784e-05,
1644
- "loss": 0.4806,
1645
- "step": 1135
1646
- },
1647
- {
1648
- "epoch": 5.601965601965602,
1649
- "grad_norm": 1.2421875,
1650
- "learning_rate": 9.596018954999953e-05,
1651
- "loss": 0.4897,
1652
- "step": 1140
1653
- },
1654
- {
1655
- "epoch": 5.6265356265356266,
1656
- "grad_norm": 1.015625,
1657
- "learning_rate": 9.510128320015224e-05,
1658
- "loss": 0.4749,
1659
- "step": 1145
1660
- },
1661
- {
1662
- "epoch": 5.651105651105651,
1663
- "grad_norm": 0.93359375,
1664
- "learning_rate": 9.424273896180482e-05,
1665
- "loss": 0.4837,
1666
- "step": 1150
1667
- },
1668
- {
1669
- "epoch": 5.675675675675675,
1670
- "grad_norm": 0.8984375,
1671
- "learning_rate": 9.338462029825886e-05,
1672
- "loss": 0.4928,
1673
- "step": 1155
1674
- },
1675
- {
1676
- "epoch": 5.7002457002457,
1677
- "grad_norm": 0.97265625,
1678
- "learning_rate": 9.252699064135758e-05,
1679
- "loss": 0.477,
1680
- "step": 1160
1681
- },
1682
- {
1683
- "epoch": 5.724815724815725,
1684
- "grad_norm": 0.87890625,
1685
- "learning_rate": 9.166991338679715e-05,
1686
- "loss": 0.4882,
1687
- "step": 1165
1688
- },
1689
- {
1690
- "epoch": 5.749385749385749,
1691
- "grad_norm": 1.078125,
1692
- "learning_rate": 9.081345188944019e-05,
1693
- "loss": 0.4902,
1694
- "step": 1170
1695
- },
1696
- {
1697
- "epoch": 5.773955773955774,
1698
- "grad_norm": 0.97265625,
1699
- "learning_rate": 8.995766945863277e-05,
1700
- "loss": 0.4873,
1701
- "step": 1175
1702
- },
1703
- {
1704
- "epoch": 5.798525798525798,
1705
- "grad_norm": 1.0625,
1706
- "learning_rate": 8.91026293535247e-05,
1707
- "loss": 0.4864,
1708
- "step": 1180
1709
- },
1710
- {
1711
- "epoch": 5.823095823095823,
1712
- "grad_norm": 1.1953125,
1713
- "learning_rate": 8.82483947783932e-05,
1714
- "loss": 0.4782,
1715
- "step": 1185
1716
- },
1717
- {
1718
- "epoch": 5.847665847665848,
1719
- "grad_norm": 0.921875,
1720
- "learning_rate": 8.739502887797107e-05,
1721
- "loss": 0.4918,
1722
- "step": 1190
1723
- },
1724
- {
1725
- "epoch": 5.872235872235873,
1726
- "grad_norm": 0.9296875,
1727
- "learning_rate": 8.654259473277892e-05,
1728
- "loss": 0.4955,
1729
- "step": 1195
1730
- },
1731
- {
1732
- "epoch": 5.896805896805897,
1733
- "grad_norm": 1.015625,
1734
- "learning_rate": 8.569115535446228e-05,
1735
- "loss": 0.4802,
1736
- "step": 1200
1737
- },
1738
- {
1739
- "epoch": 5.921375921375922,
1740
- "grad_norm": 0.8984375,
1741
- "learning_rate": 8.484077368113399e-05,
1742
- "loss": 0.4832,
1743
- "step": 1205
1744
- },
1745
- {
1746
- "epoch": 5.945945945945946,
1747
- "grad_norm": 0.87890625,
1748
- "learning_rate": 8.399151257272156e-05,
1749
- "loss": 0.4847,
1750
- "step": 1210
1751
- },
1752
- {
1753
- "epoch": 5.9705159705159705,
1754
- "grad_norm": 0.87890625,
1755
- "learning_rate": 8.314343480632078e-05,
1756
- "loss": 0.48,
1757
- "step": 1215
1758
- },
1759
- {
1760
- "epoch": 5.995085995085995,
1761
- "grad_norm": 0.87890625,
1762
- "learning_rate": 8.229660307155518e-05,
1763
- "loss": 0.4821,
1764
- "step": 1220
1765
- },
1766
- {
1767
- "epoch": 6.0,
1768
- "eval_loss": 3.3285133838653564,
1769
- "eval_runtime": 2.0506,
1770
- "eval_samples_per_second": 4.877,
1771
- "eval_steps_per_second": 0.975,
1772
- "step": 1221
1773
- },
1774
- {
1775
- "epoch": 6.019656019656019,
1776
- "grad_norm": 0.80078125,
1777
- "learning_rate": 8.145107996594206e-05,
1778
- "loss": 0.4087,
1779
- "step": 1225
1780
- },
1781
- {
1782
- "epoch": 6.044226044226044,
1783
- "grad_norm": 0.98046875,
1784
- "learning_rate": 8.060692799026522e-05,
1785
- "loss": 0.3843,
1786
- "step": 1230
1787
- },
1788
- {
1789
- "epoch": 6.068796068796069,
1790
- "grad_norm": 0.828125,
1791
- "learning_rate": 7.976420954395518e-05,
1792
- "loss": 0.3844,
1793
- "step": 1235
1794
- },
1795
- {
1796
- "epoch": 6.093366093366093,
1797
- "grad_norm": 0.984375,
1798
- "learning_rate": 7.892298692047621e-05,
1799
- "loss": 0.3909,
1800
- "step": 1240
1801
- },
1802
- {
1803
- "epoch": 6.117936117936118,
1804
- "grad_norm": 1.1171875,
1805
- "learning_rate": 7.808332230272209e-05,
1806
- "loss": 0.393,
1807
- "step": 1245
1808
- },
1809
- {
1810
- "epoch": 6.142506142506143,
1811
- "grad_norm": 0.9453125,
1812
- "learning_rate": 7.724527775841914e-05,
1813
- "loss": 0.3818,
1814
- "step": 1250
1815
- },
1816
- {
1817
- "epoch": 6.167076167076167,
1818
- "grad_norm": 0.921875,
1819
- "learning_rate": 7.64089152355385e-05,
1820
- "loss": 0.3938,
1821
- "step": 1255
1822
- },
1823
- {
1824
- "epoch": 6.191646191646192,
1825
- "grad_norm": 0.90234375,
1826
- "learning_rate": 7.55742965577169e-05,
1827
- "loss": 0.3885,
1828
- "step": 1260
1829
- },
1830
- {
1831
- "epoch": 6.216216216216216,
1832
- "grad_norm": 0.94921875,
1833
- "learning_rate": 7.474148341968652e-05,
1834
- "loss": 0.3889,
1835
- "step": 1265
1836
- },
1837
- {
1838
- "epoch": 6.240786240786241,
1839
- "grad_norm": 1.171875,
1840
- "learning_rate": 7.391053738271466e-05,
1841
- "loss": 0.3932,
1842
- "step": 1270
1843
- },
1844
- {
1845
- "epoch": 6.2653562653562656,
1846
- "grad_norm": 0.91015625,
1847
- "learning_rate": 7.308151987005326e-05,
1848
- "loss": 0.3823,
1849
- "step": 1275
1850
- },
1851
- {
1852
- "epoch": 6.2899262899262895,
1853
- "grad_norm": 0.91015625,
1854
- "learning_rate": 7.225449216239821e-05,
1855
- "loss": 0.3857,
1856
- "step": 1280
1857
- },
1858
- {
1859
- "epoch": 6.314496314496314,
1860
- "grad_norm": 0.8671875,
1861
- "learning_rate": 7.142951539335981e-05,
1862
- "loss": 0.3973,
1863
- "step": 1285
1864
- },
1865
- {
1866
- "epoch": 6.339066339066339,
1867
- "grad_norm": 0.91796875,
1868
- "learning_rate": 7.060665054494362e-05,
1869
- "loss": 0.3901,
1870
- "step": 1290
1871
- },
1872
- {
1873
- "epoch": 6.363636363636363,
1874
- "grad_norm": 0.89453125,
1875
- "learning_rate": 6.978595844304271e-05,
1876
- "loss": 0.386,
1877
- "step": 1295
1878
- },
1879
- {
1880
- "epoch": 6.388206388206388,
1881
- "grad_norm": 1.03125,
1882
- "learning_rate": 6.89674997529416e-05,
1883
- "loss": 0.3881,
1884
- "step": 1300
1885
- },
1886
- {
1887
- "epoch": 6.412776412776413,
1888
- "grad_norm": 0.89453125,
1889
- "learning_rate": 6.815133497483157e-05,
1890
- "loss": 0.4006,
1891
- "step": 1305
1892
- },
1893
- {
1894
- "epoch": 6.437346437346437,
1895
- "grad_norm": 1.0234375,
1896
- "learning_rate": 6.733752443933878e-05,
1897
- "loss": 0.397,
1898
- "step": 1310
1899
- },
1900
- {
1901
- "epoch": 6.461916461916462,
1902
- "grad_norm": 1.0703125,
1903
- "learning_rate": 6.65261283030646e-05,
1904
- "loss": 0.3972,
1905
- "step": 1315
1906
- },
1907
- {
1908
- "epoch": 6.486486486486487,
1909
- "grad_norm": 1.0390625,
1910
- "learning_rate": 6.571720654413877e-05,
1911
- "loss": 0.3965,
1912
- "step": 1320
1913
- },
1914
- {
1915
- "epoch": 6.511056511056511,
1916
- "grad_norm": 0.9296875,
1917
- "learning_rate": 6.491081895778588e-05,
1918
- "loss": 0.3961,
1919
- "step": 1325
1920
- },
1921
- {
1922
- "epoch": 6.535626535626536,
1923
- "grad_norm": 1.0625,
1924
- "learning_rate": 6.410702515190543e-05,
1925
- "loss": 0.3919,
1926
- "step": 1330
1927
- },
1928
- {
1929
- "epoch": 6.560196560196561,
1930
- "grad_norm": 0.9453125,
1931
- "learning_rate": 6.330588454266542e-05,
1932
- "loss": 0.3916,
1933
- "step": 1335
1934
- },
1935
- {
1936
- "epoch": 6.584766584766585,
1937
- "grad_norm": 0.9453125,
1938
- "learning_rate": 6.250745635011048e-05,
1939
- "loss": 0.4089,
1940
- "step": 1340
1941
- },
1942
- {
1943
- "epoch": 6.6093366093366095,
1944
- "grad_norm": 0.9140625,
1945
- "learning_rate": 6.171179959378437e-05,
1946
- "loss": 0.401,
1947
- "step": 1345
1948
- },
1949
- {
1950
- "epoch": 6.6339066339066335,
1951
- "grad_norm": 0.89453125,
1952
- "learning_rate": 6.0918973088367116e-05,
1953
- "loss": 0.3927,
1954
- "step": 1350
1955
- },
1956
- {
1957
- "epoch": 6.658476658476658,
1958
- "grad_norm": 1.0,
1959
- "learning_rate": 6.012903543932766e-05,
1960
- "loss": 0.3899,
1961
- "step": 1355
1962
- },
1963
- {
1964
- "epoch": 6.683046683046683,
1965
- "grad_norm": 0.84765625,
1966
- "learning_rate": 5.934204503859158e-05,
1967
- "loss": 0.3952,
1968
- "step": 1360
1969
- },
1970
- {
1971
- "epoch": 6.707616707616707,
1972
- "grad_norm": 0.98828125,
1973
- "learning_rate": 5.8558060060224817e-05,
1974
- "loss": 0.3917,
1975
- "step": 1365
1976
- },
1977
- {
1978
- "epoch": 6.732186732186732,
1979
- "grad_norm": 0.92578125,
1980
- "learning_rate": 5.777713845613364e-05,
1981
- "loss": 0.3878,
1982
- "step": 1370
1983
- },
1984
- {
1985
- "epoch": 6.756756756756757,
1986
- "grad_norm": 0.95703125,
1987
- "learning_rate": 5.699933795178052e-05,
1988
- "loss": 0.4066,
1989
- "step": 1375
1990
- },
1991
- {
1992
- "epoch": 6.781326781326781,
1993
- "grad_norm": 0.90625,
1994
- "learning_rate": 5.622471604191746e-05,
1995
- "loss": 0.3993,
1996
- "step": 1380
1997
- },
1998
- {
1999
- "epoch": 6.805896805896806,
2000
- "grad_norm": 0.98828125,
2001
- "learning_rate": 5.545332998633572e-05,
2002
- "loss": 0.3975,
2003
- "step": 1385
2004
- },
2005
- {
2006
- "epoch": 6.830466830466831,
2007
- "grad_norm": 0.96875,
2008
- "learning_rate": 5.46852368056334e-05,
2009
- "loss": 0.4003,
2010
- "step": 1390
2011
- },
2012
- {
2013
- "epoch": 6.855036855036855,
2014
- "grad_norm": 0.828125,
2015
- "learning_rate": 5.392049327700026e-05,
2016
- "loss": 0.3978,
2017
- "step": 1395
2018
- },
2019
- {
2020
- "epoch": 6.87960687960688,
2021
- "grad_norm": 0.90625,
2022
- "learning_rate": 5.3159155930021e-05,
2023
- "loss": 0.4017,
2024
- "step": 1400
2025
- },
2026
- {
2027
- "epoch": 6.9041769041769046,
2028
- "grad_norm": 0.90625,
2029
- "learning_rate": 5.2401281042496494e-05,
2030
- "loss": 0.3992,
2031
- "step": 1405
2032
- },
2033
- {
2034
- "epoch": 6.9287469287469285,
2035
- "grad_norm": 0.95703125,
2036
- "learning_rate": 5.164692463628378e-05,
2037
- "loss": 0.3965,
2038
- "step": 1410
2039
- },
2040
- {
2041
- "epoch": 6.953316953316953,
2042
- "grad_norm": 0.9140625,
2043
- "learning_rate": 5.0896142473154987e-05,
2044
- "loss": 0.3883,
2045
- "step": 1415
2046
- },
2047
- {
2048
- "epoch": 6.977886977886978,
2049
- "grad_norm": 0.95703125,
2050
- "learning_rate": 5.014899005067524e-05,
2051
- "loss": 0.3899,
2052
- "step": 1420
2053
- },
2054
- {
2055
- "epoch": 6.997542997542998,
2056
- "eval_loss": 3.898437023162842,
2057
- "eval_runtime": 2.0525,
2058
- "eval_samples_per_second": 4.872,
2059
- "eval_steps_per_second": 0.974,
2060
- "step": 1424
2061
- },
2062
- {
2063
- "epoch": 7.002457002457002,
2064
- "grad_norm": 1.0546875,
2065
- "learning_rate": 4.940552259810063e-05,
2066
- "loss": 0.3846,
2067
- "step": 1425
2068
- },
2069
- {
2070
- "epoch": 7.027027027027027,
2071
- "grad_norm": 0.82421875,
2072
- "learning_rate": 4.866579507229545e-05,
2073
- "loss": 0.3325,
2074
- "step": 1430
2075
- },
2076
- {
2077
- "epoch": 7.051597051597051,
2078
- "grad_norm": 1.0,
2079
- "learning_rate": 4.792986215366976e-05,
2080
- "loss": 0.3266,
2081
- "step": 1435
2082
- },
2083
- {
2084
- "epoch": 7.076167076167076,
2085
- "grad_norm": 1.0078125,
2086
- "learning_rate": 4.7197778242137755e-05,
2087
- "loss": 0.3295,
2088
- "step": 1440
2089
- },
2090
- {
2091
- "epoch": 7.100737100737101,
2092
- "grad_norm": 0.80078125,
2093
- "learning_rate": 4.646959745309609e-05,
2094
- "loss": 0.3279,
2095
- "step": 1445
2096
- },
2097
- {
2098
- "epoch": 7.125307125307125,
2099
- "grad_norm": 0.96875,
2100
- "learning_rate": 4.574537361342407e-05,
2101
- "loss": 0.324,
2102
- "step": 1450
2103
- },
2104
- {
2105
- "epoch": 7.14987714987715,
2106
- "grad_norm": 0.87109375,
2107
- "learning_rate": 4.502516025750455e-05,
2108
- "loss": 0.319,
2109
- "step": 1455
2110
- },
2111
- {
2112
- "epoch": 7.174447174447175,
2113
- "grad_norm": 0.8125,
2114
- "learning_rate": 4.430901062326681e-05,
2115
- "loss": 0.3242,
2116
- "step": 1460
2117
- },
2118
- {
2119
- "epoch": 7.199017199017199,
2120
- "grad_norm": 0.8515625,
2121
- "learning_rate": 4.359697764825123e-05,
2122
- "loss": 0.3172,
2123
- "step": 1465
2124
- },
2125
- {
2126
- "epoch": 7.223587223587224,
2127
- "grad_norm": 0.9140625,
2128
- "learning_rate": 4.288911396569599e-05,
2129
- "loss": 0.3275,
2130
- "step": 1470
2131
- },
2132
- {
2133
- "epoch": 7.2481572481572485,
2134
- "grad_norm": 0.8828125,
2135
- "learning_rate": 4.21854719006467e-05,
2136
- "loss": 0.332,
2137
- "step": 1475
2138
- },
2139
- {
2140
- "epoch": 7.2727272727272725,
2141
- "grad_norm": 1.015625,
2142
- "learning_rate": 4.148610346608837e-05,
2143
- "loss": 0.3359,
2144
- "step": 1480
2145
- },
2146
- {
2147
- "epoch": 7.297297297297297,
2148
- "grad_norm": 0.87109375,
2149
- "learning_rate": 4.079106035910073e-05,
2150
- "loss": 0.3242,
2151
- "step": 1485
2152
- },
2153
- {
2154
- "epoch": 7.321867321867322,
2155
- "grad_norm": 0.86328125,
2156
- "learning_rate": 4.010039395703664e-05,
2157
- "loss": 0.3273,
2158
- "step": 1490
2159
- },
2160
- {
2161
- "epoch": 7.346437346437346,
2162
- "grad_norm": 0.9140625,
2163
- "learning_rate": 3.94141553137245e-05,
2164
- "loss": 0.3274,
2165
- "step": 1495
2166
- },
2167
- {
2168
- "epoch": 7.371007371007371,
2169
- "grad_norm": 0.88671875,
2170
- "learning_rate": 3.873239515569429e-05,
2171
- "loss": 0.3266,
2172
- "step": 1500
2173
- },
2174
- {
2175
- "epoch": 7.395577395577396,
2176
- "grad_norm": 0.84765625,
2177
- "learning_rate": 3.80551638784277e-05,
2178
- "loss": 0.3363,
2179
- "step": 1505
2180
- },
2181
- {
2182
- "epoch": 7.42014742014742,
2183
- "grad_norm": 0.87109375,
2184
- "learning_rate": 3.738251154263333e-05,
2185
- "loss": 0.335,
2186
- "step": 1510
2187
- },
2188
- {
2189
- "epoch": 7.444717444717445,
2190
- "grad_norm": 0.84765625,
2191
- "learning_rate": 3.671448787054571e-05,
2192
- "loss": 0.3305,
2193
- "step": 1515
2194
- },
2195
- {
2196
- "epoch": 7.469287469287469,
2197
- "grad_norm": 0.8515625,
2198
- "learning_rate": 3.605114224225028e-05,
2199
- "loss": 0.332,
2200
- "step": 1520
2201
- },
2202
- {
2203
- "epoch": 7.493857493857494,
2204
- "grad_norm": 0.89453125,
2205
- "learning_rate": 3.5392523692033006e-05,
2206
- "loss": 0.3261,
2207
- "step": 1525
2208
- },
2209
- {
2210
- "epoch": 7.518427518427519,
2211
- "grad_norm": 1.09375,
2212
- "learning_rate": 3.473868090475574e-05,
2213
- "loss": 0.3363,
2214
- "step": 1530
2215
- },
2216
- {
2217
- "epoch": 7.542997542997543,
2218
- "grad_norm": 0.88671875,
2219
- "learning_rate": 3.408966221225773e-05,
2220
- "loss": 0.3239,
2221
- "step": 1535
2222
- },
2223
- {
2224
- "epoch": 7.5675675675675675,
2225
- "grad_norm": 0.875,
2226
- "learning_rate": 3.3445515589782574e-05,
2227
- "loss": 0.3301,
2228
- "step": 1540
2229
- },
2230
- {
2231
- "epoch": 7.592137592137592,
2232
- "grad_norm": 0.875,
2233
- "learning_rate": 3.2806288652432174e-05,
2234
- "loss": 0.3228,
2235
- "step": 1545
2236
- },
2237
- {
2238
- "epoch": 7.616707616707616,
2239
- "grad_norm": 0.8828125,
2240
- "learning_rate": 3.217202865164697e-05,
2241
- "loss": 0.3389,
2242
- "step": 1550
2243
- },
2244
- {
2245
- "epoch": 7.641277641277641,
2246
- "grad_norm": 0.859375,
2247
- "learning_rate": 3.154278247171314e-05,
2248
- "loss": 0.3302,
2249
- "step": 1555
2250
- },
2251
- {
2252
- "epoch": 7.665847665847666,
2253
- "grad_norm": 0.89453125,
2254
- "learning_rate": 3.09185966262968e-05,
2255
- "loss": 0.3245,
2256
- "step": 1560
2257
- },
2258
- {
2259
- "epoch": 7.69041769041769,
2260
- "grad_norm": 0.86328125,
2261
- "learning_rate": 3.0299517255005937e-05,
2262
- "loss": 0.3306,
2263
- "step": 1565
2264
- },
2265
- {
2266
- "epoch": 7.714987714987715,
2267
- "grad_norm": 0.90625,
2268
- "learning_rate": 2.9685590119979688e-05,
2269
- "loss": 0.3322,
2270
- "step": 1570
2271
- },
2272
- {
2273
- "epoch": 7.739557739557739,
2274
- "grad_norm": 0.8828125,
2275
- "learning_rate": 2.9076860602505564e-05,
2276
- "loss": 0.3327,
2277
- "step": 1575
2278
- },
2279
- {
2280
- "epoch": 7.764127764127764,
2281
- "grad_norm": 0.953125,
2282
- "learning_rate": 2.8473373699664997e-05,
2283
- "loss": 0.3342,
2284
- "step": 1580
2285
- },
2286
- {
2287
- "epoch": 7.788697788697789,
2288
- "grad_norm": 0.89453125,
2289
- "learning_rate": 2.7875174021007e-05,
2290
- "loss": 0.3318,
2291
- "step": 1585
2292
- },
2293
- {
2294
- "epoch": 7.813267813267814,
2295
- "grad_norm": 0.85546875,
2296
- "learning_rate": 2.728230578525086e-05,
2297
- "loss": 0.3357,
2298
- "step": 1590
2299
- },
2300
- {
2301
- "epoch": 7.837837837837838,
2302
- "grad_norm": 0.8984375,
2303
- "learning_rate": 2.669481281701739e-05,
2304
- "loss": 0.3345,
2305
- "step": 1595
2306
- },
2307
- {
2308
- "epoch": 7.862407862407863,
2309
- "grad_norm": 0.94921875,
2310
- "learning_rate": 2.6112738543589312e-05,
2311
- "loss": 0.3324,
2312
- "step": 1600
2313
- },
2314
- {
2315
- "epoch": 7.886977886977887,
2316
- "grad_norm": 0.86328125,
2317
- "learning_rate": 2.553612599170143e-05,
2318
- "loss": 0.3278,
2319
- "step": 1605
2320
- },
2321
- {
2322
- "epoch": 7.9115479115479115,
2323
- "grad_norm": 0.8046875,
2324
- "learning_rate": 2.496501778435977e-05,
2325
- "loss": 0.323,
2326
- "step": 1610
2327
- },
2328
- {
2329
- "epoch": 7.936117936117936,
2330
- "grad_norm": 0.953125,
2331
- "learning_rate": 2.4399456137691147e-05,
2332
- "loss": 0.3364,
2333
- "step": 1615
2334
- },
2335
- {
2336
- "epoch": 7.96068796068796,
2337
- "grad_norm": 0.85546875,
2338
- "learning_rate": 2.3839482857822458e-05,
2339
- "loss": 0.3348,
2340
- "step": 1620
2341
- },
2342
- {
2343
- "epoch": 7.985257985257985,
2344
- "grad_norm": 0.92578125,
2345
- "learning_rate": 2.328513933779034e-05,
2346
- "loss": 0.3321,
2347
- "step": 1625
2348
- },
2349
- {
2350
- "epoch": 8.0,
2351
- "eval_loss": 4.43484354019165,
2352
- "eval_runtime": 2.0469,
2353
- "eval_samples_per_second": 4.885,
2354
- "eval_steps_per_second": 0.977,
2355
- "step": 1628
2356
- },
2357
- {
2358
- "epoch": 8.00982800982801,
2359
- "grad_norm": 0.703125,
2360
- "learning_rate": 2.2736466554481617e-05,
2361
- "loss": 0.3114,
2362
- "step": 1630
2363
- },
2364
- {
2365
- "epoch": 8.034398034398034,
2366
- "grad_norm": 0.78125,
2367
- "learning_rate": 2.2193505065604014e-05,
2368
- "loss": 0.3037,
2369
- "step": 1635
2370
- },
2371
- {
2372
- "epoch": 8.058968058968059,
2373
- "grad_norm": 0.78515625,
2374
- "learning_rate": 2.1656295006688353e-05,
2375
- "loss": 0.2959,
2376
- "step": 1640
2377
- },
2378
- {
2379
- "epoch": 8.083538083538084,
2380
- "grad_norm": 0.88671875,
2381
- "learning_rate": 2.1124876088121692e-05,
2382
- "loss": 0.3102,
2383
- "step": 1645
2384
- },
2385
- {
2386
- "epoch": 8.108108108108109,
2387
- "grad_norm": 0.83984375,
2388
- "learning_rate": 2.0599287592211968e-05,
2389
- "loss": 0.2975,
2390
- "step": 1650
2391
- },
2392
- {
2393
- "epoch": 8.132678132678132,
2394
- "grad_norm": 0.82421875,
2395
- "learning_rate": 2.0079568370284128e-05,
2396
- "loss": 0.2934,
2397
- "step": 1655
2398
- },
2399
- {
2400
- "epoch": 8.157248157248157,
2401
- "grad_norm": 0.84765625,
2402
- "learning_rate": 1.956575683980846e-05,
2403
- "loss": 0.299,
2404
- "step": 1660
2405
- },
2406
- {
2407
- "epoch": 8.181818181818182,
2408
- "grad_norm": 0.81640625,
2409
- "learning_rate": 1.9057890981560677e-05,
2410
- "loss": 0.2952,
2411
- "step": 1665
2412
- },
2413
- {
2414
- "epoch": 8.206388206388207,
2415
- "grad_norm": 0.8046875,
2416
- "learning_rate": 1.85560083368143e-05,
2417
- "loss": 0.3067,
2418
- "step": 1670
2419
- },
2420
- {
2421
- "epoch": 8.230958230958231,
2422
- "grad_norm": 0.8203125,
2423
- "learning_rate": 1.806014600456588e-05,
2424
- "loss": 0.2978,
2425
- "step": 1675
2426
- },
2427
- {
2428
- "epoch": 8.255528255528256,
2429
- "grad_norm": 0.80078125,
2430
- "learning_rate": 1.757034063879235e-05,
2431
- "loss": 0.2973,
2432
- "step": 1680
2433
- },
2434
- {
2435
- "epoch": 8.28009828009828,
2436
- "grad_norm": 0.796875,
2437
- "learning_rate": 1.708662844574178e-05,
2438
- "loss": 0.2983,
2439
- "step": 1685
2440
- },
2441
- {
2442
- "epoch": 8.304668304668304,
2443
- "grad_norm": 0.78125,
2444
- "learning_rate": 1.6609045181256976e-05,
2445
- "loss": 0.2991,
2446
- "step": 1690
2447
- },
2448
- {
2449
- "epoch": 8.32923832923833,
2450
- "grad_norm": 0.8203125,
2451
- "learning_rate": 1.61376261481323e-05,
2452
- "loss": 0.3102,
2453
- "step": 1695
2454
- },
2455
- {
2456
- "epoch": 8.353808353808354,
2457
- "grad_norm": 0.8828125,
2458
- "learning_rate": 1.5672406193504384e-05,
2459
- "loss": 0.2984,
2460
- "step": 1700
2461
- },
2462
- {
2463
- "epoch": 8.378378378378379,
2464
- "grad_norm": 0.7578125,
2465
- "learning_rate": 1.5213419706275878e-05,
2466
- "loss": 0.298,
2467
- "step": 1705
2468
- },
2469
- {
2470
- "epoch": 8.402948402948404,
2471
- "grad_norm": 0.8046875,
2472
- "learning_rate": 1.4760700614573731e-05,
2473
- "loss": 0.3021,
2474
- "step": 1710
2475
- },
2476
- {
2477
- "epoch": 8.427518427518427,
2478
- "grad_norm": 0.84375,
2479
- "learning_rate": 1.4314282383241096e-05,
2480
- "loss": 0.3068,
2481
- "step": 1715
2482
- },
2483
- {
2484
- "epoch": 8.452088452088452,
2485
- "grad_norm": 0.82421875,
2486
- "learning_rate": 1.3874198011363582e-05,
2487
- "loss": 0.3038,
2488
- "step": 1720
2489
- },
2490
- {
2491
- "epoch": 8.476658476658477,
2492
- "grad_norm": 0.8359375,
2493
- "learning_rate": 1.3440480029830127e-05,
2494
- "loss": 0.3024,
2495
- "step": 1725
2496
- },
2497
- {
2498
- "epoch": 8.501228501228502,
2499
- "grad_norm": 0.86328125,
2500
- "learning_rate": 1.301316049892818e-05,
2501
- "loss": 0.3018,
2502
- "step": 1730
2503
- },
2504
- {
2505
- "epoch": 8.525798525798526,
2506
- "grad_norm": 0.87890625,
2507
- "learning_rate": 1.2592271005973888e-05,
2508
- "loss": 0.3034,
2509
- "step": 1735
2510
- },
2511
- {
2512
- "epoch": 8.55036855036855,
2513
- "grad_norm": 0.8046875,
2514
- "learning_rate": 1.2177842662977135e-05,
2515
- "loss": 0.2999,
2516
- "step": 1740
2517
- },
2518
- {
2519
- "epoch": 8.574938574938574,
2520
- "grad_norm": 0.91796875,
2521
- "learning_rate": 1.1769906104341832e-05,
2522
- "loss": 0.2997,
2523
- "step": 1745
2524
- },
2525
- {
2526
- "epoch": 8.5995085995086,
2527
- "grad_norm": 0.8203125,
2528
- "learning_rate": 1.136849148460125e-05,
2529
- "loss": 0.3042,
2530
- "step": 1750
2531
- },
2532
- {
2533
- "epoch": 8.624078624078624,
2534
- "grad_norm": 0.88671875,
2535
- "learning_rate": 1.0973628476189257e-05,
2536
- "loss": 0.2933,
2537
- "step": 1755
2538
- },
2539
- {
2540
- "epoch": 8.64864864864865,
2541
- "grad_norm": 0.8046875,
2542
- "learning_rate": 1.0585346267246743e-05,
2543
- "loss": 0.2983,
2544
- "step": 1760
2545
- },
2546
- {
2547
- "epoch": 8.673218673218674,
2548
- "grad_norm": 0.7890625,
2549
- "learning_rate": 1.0203673559464089e-05,
2550
- "loss": 0.2963,
2551
- "step": 1765
2552
- },
2553
- {
2554
- "epoch": 8.697788697788697,
2555
- "grad_norm": 0.83984375,
2556
- "learning_rate": 9.82863856595968e-06,
2557
- "loss": 0.2952,
2558
- "step": 1770
2559
- },
2560
- {
2561
- "epoch": 8.722358722358722,
2562
- "grad_norm": 0.8359375,
2563
- "learning_rate": 9.460269009194167e-06,
2564
- "loss": 0.304,
2565
- "step": 1775
2566
- },
2567
- {
2568
- "epoch": 8.746928746928747,
2569
- "grad_norm": 0.84765625,
2570
- "learning_rate": 9.098592118921435e-06,
2571
- "loss": 0.3037,
2572
- "step": 1780
2573
- },
2574
- {
2575
- "epoch": 8.771498771498772,
2576
- "grad_norm": 0.79296875,
2577
- "learning_rate": 8.74363463017569e-06,
2578
- "loss": 0.3058,
2579
- "step": 1785
2580
- },
2581
- {
2582
- "epoch": 8.796068796068797,
2583
- "grad_norm": 1.03125,
2584
- "learning_rate": 8.395422781295192e-06,
2585
- "loss": 0.3031,
2586
- "step": 1790
2587
- },
2588
- {
2589
- "epoch": 8.82063882063882,
2590
- "grad_norm": 0.875,
2591
- "learning_rate": 8.053982311982867e-06,
2592
- "loss": 0.3017,
2593
- "step": 1795
2594
- },
2595
- {
2596
- "epoch": 8.845208845208845,
2597
- "grad_norm": 0.9453125,
2598
- "learning_rate": 7.719338461403435e-06,
2599
- "loss": 0.2978,
2600
- "step": 1800
2601
- },
2602
- {
2603
- "epoch": 8.86977886977887,
2604
- "grad_norm": 0.82421875,
2605
- "learning_rate": 7.3915159663179075e-06,
2606
- "loss": 0.2905,
2607
- "step": 1805
2608
- },
2609
- {
2610
- "epoch": 8.894348894348894,
2611
- "grad_norm": 0.81640625,
2612
- "learning_rate": 7.070539059254977e-06,
2613
- "loss": 0.2957,
2614
- "step": 1810
2615
- },
2616
- {
2617
- "epoch": 8.91891891891892,
2618
- "grad_norm": 0.7890625,
2619
- "learning_rate": 6.756431466719737e-06,
2620
- "loss": 0.2988,
2621
- "step": 1815
2622
- },
2623
- {
2624
- "epoch": 8.943488943488944,
2625
- "grad_norm": 0.84375,
2626
- "learning_rate": 6.4492164074399065e-06,
2627
- "loss": 0.3019,
2628
- "step": 1820
2629
- },
2630
- {
2631
- "epoch": 8.968058968058967,
2632
- "grad_norm": 0.8359375,
2633
- "learning_rate": 6.148916590649434e-06,
2634
- "loss": 0.3051,
2635
- "step": 1825
2636
- },
2637
- {
2638
- "epoch": 8.992628992628992,
2639
- "grad_norm": 0.859375,
2640
- "learning_rate": 5.8555542144098865e-06,
2641
- "loss": 0.3089,
2642
- "step": 1830
2643
- },
2644
- {
2645
- "epoch": 8.997542997542997,
2646
- "eval_loss": 4.689078330993652,
2647
- "eval_runtime": 2.054,
2648
- "eval_samples_per_second": 4.868,
2649
- "eval_steps_per_second": 0.974,
2650
- "step": 1831
2651
- },
2652
- {
2653
- "epoch": 9.017199017199017,
2654
- "grad_norm": 0.78515625,
2655
- "learning_rate": 5.569150963969494e-06,
2656
- "loss": 0.2973,
2657
- "step": 1835
2658
- },
2659
- {
2660
- "epoch": 9.041769041769042,
2661
- "grad_norm": 0.80078125,
2662
- "learning_rate": 5.289728010160366e-06,
2663
- "loss": 0.3,
2664
- "step": 1840
2665
- },
2666
- {
2667
- "epoch": 9.066339066339067,
2668
- "grad_norm": 0.7578125,
2669
- "learning_rate": 5.0173060078333225e-06,
2670
- "loss": 0.299,
2671
- "step": 1845
2672
- },
2673
- {
2674
- "epoch": 9.090909090909092,
2675
- "grad_norm": 0.796875,
2676
- "learning_rate": 4.7519050943312325e-06,
2677
- "loss": 0.2971,
2678
- "step": 1850
2679
- },
2680
- {
2681
- "epoch": 9.115479115479115,
2682
- "grad_norm": 0.78125,
2683
- "learning_rate": 4.493544888000467e-06,
2684
- "loss": 0.2968,
2685
- "step": 1855
2686
- },
2687
- {
2688
- "epoch": 9.14004914004914,
2689
- "grad_norm": 0.765625,
2690
- "learning_rate": 4.242244486740643e-06,
2691
- "loss": 0.295,
2692
- "step": 1860
2693
- },
2694
- {
2695
- "epoch": 9.164619164619165,
2696
- "grad_norm": 0.8125,
2697
- "learning_rate": 3.99802246659301e-06,
2698
- "loss": 0.2911,
2699
- "step": 1865
2700
- },
2701
- {
2702
- "epoch": 9.18918918918919,
2703
- "grad_norm": 0.76953125,
2704
- "learning_rate": 3.760896880367215e-06,
2705
- "loss": 0.2916,
2706
- "step": 1870
2707
- },
2708
- {
2709
- "epoch": 9.213759213759214,
2710
- "grad_norm": 0.7890625,
2711
- "learning_rate": 3.530885256306915e-06,
2712
- "loss": 0.2998,
2713
- "step": 1875
2714
- },
2715
- {
2716
- "epoch": 9.238329238329237,
2717
- "grad_norm": 0.8046875,
2718
- "learning_rate": 3.308004596794101e-06,
2719
- "loss": 0.2948,
2720
- "step": 1880
2721
- },
2722
- {
2723
- "epoch": 9.262899262899262,
2724
- "grad_norm": 0.81640625,
2725
- "learning_rate": 3.092271377092215e-06,
2726
- "loss": 0.2917,
2727
- "step": 1885
2728
- },
2729
- {
2730
- "epoch": 9.287469287469287,
2731
- "grad_norm": 0.75,
2732
- "learning_rate": 2.8837015441283586e-06,
2733
- "loss": 0.295,
2734
- "step": 1890
2735
- },
2736
- {
2737
- "epoch": 9.312039312039312,
2738
- "grad_norm": 0.74609375,
2739
- "learning_rate": 2.682310515314512e-06,
2740
- "loss": 0.2928,
2741
- "step": 1895
2742
- },
2743
- {
2744
- "epoch": 9.336609336609337,
2745
- "grad_norm": 0.8046875,
2746
- "learning_rate": 2.488113177407869e-06,
2747
- "loss": 0.2909,
2748
- "step": 1900
2749
- },
2750
- {
2751
- "epoch": 9.361179361179362,
2752
- "grad_norm": 0.75,
2753
- "learning_rate": 2.3011238854103947e-06,
2754
- "loss": 0.2929,
2755
- "step": 1905
2756
- },
2757
- {
2758
- "epoch": 9.385749385749385,
2759
- "grad_norm": 0.796875,
2760
- "learning_rate": 2.1213564615077065e-06,
2761
- "loss": 0.2936,
2762
- "step": 1910
2763
- },
2764
- {
2765
- "epoch": 9.41031941031941,
2766
- "grad_norm": 0.79296875,
2767
- "learning_rate": 1.9488241940473828e-06,
2768
- "loss": 0.2941,
2769
- "step": 1915
2770
- },
2771
- {
2772
- "epoch": 9.434889434889435,
2773
- "grad_norm": 0.76171875,
2774
- "learning_rate": 1.783539836556669e-06,
2775
- "loss": 0.2941,
2776
- "step": 1920
2777
- },
2778
- {
2779
- "epoch": 9.45945945945946,
2780
- "grad_norm": 0.78515625,
2781
- "learning_rate": 1.6255156067997323e-06,
2782
- "loss": 0.2869,
2783
- "step": 1925
2784
- },
2785
- {
2786
- "epoch": 9.484029484029485,
2787
- "grad_norm": 0.80859375,
2788
- "learning_rate": 1.474763185874517e-06,
2789
- "loss": 0.3029,
2790
- "step": 1930
2791
- },
2792
- {
2793
- "epoch": 9.50859950859951,
2794
- "grad_norm": 0.80859375,
2795
- "learning_rate": 1.3312937173493577e-06,
2796
- "loss": 0.2943,
2797
- "step": 1935
2798
- },
2799
- {
2800
- "epoch": 9.533169533169533,
2801
- "grad_norm": 0.83984375,
2802
- "learning_rate": 1.19511780643915e-06,
2803
- "loss": 0.2963,
2804
- "step": 1940
2805
- },
2806
- {
2807
- "epoch": 9.557739557739557,
2808
- "grad_norm": 0.765625,
2809
- "learning_rate": 1.066245519221465e-06,
2810
- "loss": 0.3002,
2811
- "step": 1945
2812
- },
2813
- {
2814
- "epoch": 9.582309582309582,
2815
- "grad_norm": 0.76171875,
2816
- "learning_rate": 9.446863818924679e-07,
2817
- "loss": 0.2988,
2818
- "step": 1950
2819
- },
2820
- {
2821
- "epoch": 9.606879606879607,
2822
- "grad_norm": 0.77734375,
2823
- "learning_rate": 8.304493800627589e-07,
2824
- "loss": 0.297,
2825
- "step": 1955
2826
- },
2827
- {
2828
- "epoch": 9.631449631449632,
2829
- "grad_norm": 0.75,
2830
- "learning_rate": 7.235429580931152e-07,
2831
- "loss": 0.2902,
2832
- "step": 1960
2833
- },
2834
- {
2835
- "epoch": 9.656019656019655,
2836
- "grad_norm": 0.859375,
2837
- "learning_rate": 6.239750184703464e-07,
2838
- "loss": 0.2933,
2839
- "step": 1965
2840
- },
2841
- {
2842
- "epoch": 9.68058968058968,
2843
- "grad_norm": 0.76953125,
2844
- "learning_rate": 5.317529212230721e-07,
2845
- "loss": 0.2943,
2846
- "step": 1970
2847
- },
2848
- {
2849
- "epoch": 9.705159705159705,
2850
- "grad_norm": 0.78125,
2851
- "learning_rate": 4.4688348337774686e-07,
2852
- "loss": 0.3049,
2853
- "step": 1975
2854
- },
2855
- {
2856
- "epoch": 9.72972972972973,
2857
- "grad_norm": 0.83984375,
2858
- "learning_rate": 3.693729784546962e-07,
2859
- "loss": 0.2876,
2860
- "step": 1980
2861
- },
2862
- {
2863
- "epoch": 9.754299754299755,
2864
- "grad_norm": 0.78515625,
2865
- "learning_rate": 2.9922713600439854e-07,
2866
- "loss": 0.2971,
2867
- "step": 1985
2868
- },
2869
- {
2870
- "epoch": 9.77886977886978,
2871
- "grad_norm": 0.765625,
2872
- "learning_rate": 2.3645114118395762e-07,
2873
- "loss": 0.288,
2874
- "step": 1990
2875
- },
2876
- {
2877
- "epoch": 9.803439803439803,
2878
- "grad_norm": 0.84765625,
2879
- "learning_rate": 1.8104963437381993e-07,
2880
- "loss": 0.2951,
2881
- "step": 1995
2882
- },
2883
- {
2884
- "epoch": 9.828009828009828,
2885
- "grad_norm": 0.7578125,
2886
- "learning_rate": 1.3302671083474938e-07,
2887
- "loss": 0.2927,
2888
- "step": 2000
2889
- },
2890
- {
2891
- "epoch": 9.852579852579852,
2892
- "grad_norm": 0.78515625,
2893
- "learning_rate": 9.238592040512472e-08,
2894
- "loss": 0.2932,
2895
- "step": 2005
2896
- },
2897
- {
2898
- "epoch": 9.877149877149877,
2899
- "grad_norm": 0.75390625,
2900
- "learning_rate": 5.913026723850523e-08,
2901
- "loss": 0.2913,
2902
- "step": 2010
2903
- },
2904
- {
2905
- "epoch": 9.901719901719902,
2906
- "grad_norm": 0.8046875,
2907
- "learning_rate": 3.3262209581619297e-08,
2908
- "loss": 0.2997,
2909
- "step": 2015
2910
- },
2911
- {
2912
- "epoch": 9.926289926289925,
2913
- "grad_norm": 0.85546875,
2914
- "learning_rate": 1.4783659592576548e-08,
2915
- "loss": 0.2977,
2916
- "step": 2020
2917
- },
2918
- {
2919
- "epoch": 9.95085995085995,
2920
- "grad_norm": 0.80078125,
2921
- "learning_rate": 3.6959831996030704e-09,
2922
- "loss": 0.2988,
2923
- "step": 2025
2924
- },
2925
- {
2926
- "epoch": 9.975429975429975,
2927
- "grad_norm": 0.84375,
2928
- "learning_rate": 0.0,
2929
- "loss": 0.3016,
2930
- "step": 2030
2931
- },
2932
- {
2933
- "epoch": 9.975429975429975,
2934
- "eval_loss": 4.717045307159424,
2935
- "eval_runtime": 2.0451,
2936
- "eval_samples_per_second": 4.89,
2937
- "eval_steps_per_second": 0.978,
2938
- "step": 2030
2939
  },
2940
  {
2941
- "epoch": 9.975429975429975,
2942
- "step": 2030,
2943
- "total_flos": 1.5518062706111283e+18,
2944
- "train_loss": 1.3545548074938394,
2945
- "train_runtime": 13205.4864,
2946
- "train_samples_per_second": 2.465,
2947
- "train_steps_per_second": 0.154
2948
  }
2949
  ],
2950
  "logging_steps": 5,
2951
- "max_steps": 2030,
2952
  "num_input_tokens_seen": 0,
2953
  "num_train_epochs": 10,
2954
  "save_steps": 100,
2955
- "total_flos": 1.5518062706111283e+18,
2956
  "train_batch_size": 4,
2957
  "trial_name": null,
2958
  "trial_params": null
 
1
  {
2
  "best_metric": null,
3
  "best_model_checkpoint": null,
4
+ "epoch": 10.0,
5
  "eval_steps": 500,
6
+ "global_step": 550,
7
  "is_hyper_param_search": false,
8
  "is_local_process_zero": true,
9
  "is_world_process_zero": true,
10
  "log_history": [
11
  {
12
+ "epoch": 0.01818181818181818,
13
+ "grad_norm": 251.0,
14
+ "learning_rate": 3.636363636363636e-06,
15
+ "loss": 46.8778,
16
  "step": 1
17
  },
18
  {
19
+ "epoch": 0.09090909090909091,
20
+ "grad_norm": 239.0,
21
+ "learning_rate": 1.8181818181818182e-05,
22
+ "loss": 46.8224,
23
  "step": 5
24
  },
25
  {
26
+ "epoch": 0.18181818181818182,
27
+ "grad_norm": 106.5,
28
+ "learning_rate": 3.6363636363636364e-05,
29
+ "loss": 39.5877,
30
  "step": 10
31
  },
32
  {
33
+ "epoch": 0.2727272727272727,
34
+ "grad_norm": 19.75,
35
+ "learning_rate": 5.4545454545454546e-05,
36
+ "loss": 28.2487,
37
  "step": 15
38
  },
39
  {
40
+ "epoch": 0.36363636363636365,
41
+ "grad_norm": 15.8125,
42
+ "learning_rate": 7.272727272727273e-05,
43
+ "loss": 23.0407,
44
  "step": 20
45
  },
46
  {
47
+ "epoch": 0.45454545454545453,
48
+ "grad_norm": 5.5,
49
+ "learning_rate": 9.090909090909092e-05,
50
+ "loss": 20.515,
51
  "step": 25
52
  },
53
  {
54
+ "epoch": 0.5454545454545454,
55
+ "grad_norm": 4.1875,
56
+ "learning_rate": 0.00010909090909090909,
57
+ "loss": 19.5202,
58
  "step": 30
59
  },
60
  {
61
+ "epoch": 0.6363636363636364,
62
+ "grad_norm": 5.75,
63
+ "learning_rate": 0.00012727272727272728,
64
+ "loss": 18.2599,
65
  "step": 35
66
  },
67
  {
68
+ "epoch": 0.7272727272727273,
69
+ "grad_norm": 10.8125,
70
+ "learning_rate": 0.00014545454545454546,
71
+ "loss": 16.923,
72
  "step": 40
73
  },
74
  {
75
+ "epoch": 0.8181818181818182,
76
+ "grad_norm": 21.25,
77
+ "learning_rate": 0.00016363636363636366,
78
+ "loss": 14.3261,
79
  "step": 45
80
  },
81
  {
82
+ "epoch": 0.9090909090909091,
83
+ "grad_norm": 29.625,
84
+ "learning_rate": 0.00018181818181818183,
85
+ "loss": 9.0683,
86
  "step": 50
87
  },
88
  {
89
+ "epoch": 1.0,
90
+ "grad_norm": 9.625,
91
+ "learning_rate": 0.0002,
92
+ "loss": 3.5798,
93
+ "step": 55
94
+ },
95
+ {
96
+ "epoch": 1.0,
97
+ "eval_loss": 4.059033393859863,
98
+ "eval_runtime": 0.2418,
99
+ "eval_samples_per_second": 41.356,
100
+ "eval_steps_per_second": 4.136,
101
  "step": 55
102
  },
103
  {
104
+ "epoch": 1.0909090909090908,
105
+ "grad_norm": 10.125,
106
+ "learning_rate": 0.00019994965423831854,
107
+ "loss": 2.5621,
108
  "step": 60
109
  },
110
  {
111
+ "epoch": 1.1818181818181819,
112
+ "grad_norm": 7.4375,
113
+ "learning_rate": 0.00019979866764718843,
114
+ "loss": 2.0701,
115
  "step": 65
116
  },
117
  {
118
+ "epoch": 1.2727272727272727,
119
+ "grad_norm": 20.125,
120
+ "learning_rate": 0.00019954719225730847,
121
+ "loss": 1.8568,
122
  "step": 70
123
  },
124
  {
125
+ "epoch": 1.3636363636363638,
126
+ "grad_norm": 0.96875,
127
+ "learning_rate": 0.00019919548128307954,
128
+ "loss": 1.6836,
129
  "step": 75
130
  },
131
  {
132
+ "epoch": 1.4545454545454546,
133
+ "grad_norm": 1.078125,
134
+ "learning_rate": 0.00019874388886763944,
135
+ "loss": 1.5757,
136
  "step": 80
137
  },
138
  {
139
+ "epoch": 1.5454545454545454,
140
+ "grad_norm": 1.171875,
141
+ "learning_rate": 0.00019819286972627066,
142
+ "loss": 1.52,
143
  "step": 85
144
  },
145
  {
146
+ "epoch": 1.6363636363636362,
147
+ "grad_norm": 1.3125,
148
+ "learning_rate": 0.00019754297868854073,
149
+ "loss": 1.4757,
150
  "step": 90
151
  },
152
  {
153
+ "epoch": 1.7272727272727273,
154
+ "grad_norm": 3.46875,
155
+ "learning_rate": 0.00019679487013963564,
156
+ "loss": 1.4221,
157
  "step": 95
158
  },
159
  {
160
+ "epoch": 1.8181818181818183,
161
+ "grad_norm": 1.6796875,
162
+ "learning_rate": 0.00019594929736144976,
163
+ "loss": 1.4012,
164
  "step": 100
165
  },
166
  {
167
+ "epoch": 1.9090909090909092,
168
+ "grad_norm": 1.671875,
169
+ "learning_rate": 0.00019500711177409454,
170
+ "loss": 1.3672,
171
  "step": 105
172
  },
173
  {
174
+ "epoch": 2.0,
175
+ "grad_norm": 0.6953125,
176
+ "learning_rate": 0.00019396926207859084,
177
+ "loss": 1.346,
178
+ "step": 110
179
+ },
180
+ {
181
+ "epoch": 2.0,
182
+ "eval_loss": 2.6943764686584473,
183
+ "eval_runtime": 0.2333,
184
+ "eval_samples_per_second": 42.871,
185
+ "eval_steps_per_second": 4.287,
186
  "step": 110
187
  },
188
  {
189
+ "epoch": 2.090909090909091,
190
+ "grad_norm": 1.484375,
191
+ "learning_rate": 0.00019283679330160726,
192
+ "loss": 1.3092,
193
  "step": 115
194
  },
195
  {
196
+ "epoch": 2.1818181818181817,
197
+ "grad_norm": 1.8828125,
198
+ "learning_rate": 0.00019161084574320696,
199
+ "loss": 1.3052,
200
  "step": 120
201
  },
202
  {
203
+ "epoch": 2.2727272727272725,
204
+ "grad_norm": 0.625,
205
+ "learning_rate": 0.00019029265382866214,
206
+ "loss": 1.2724,
207
  "step": 125
208
  },
209
  {
210
+ "epoch": 2.3636363636363638,
211
+ "grad_norm": 1.1171875,
212
+ "learning_rate": 0.00018888354486549237,
213
+ "loss": 1.2612,
214
  "step": 130
215
  },
216
  {
217
+ "epoch": 2.4545454545454546,
218
+ "grad_norm": 1.1953125,
219
+ "learning_rate": 0.00018738493770697852,
220
+ "loss": 1.2532,
221
  "step": 135
222
  },
223
  {
224
+ "epoch": 2.5454545454545454,
225
+ "grad_norm": 4.25,
226
+ "learning_rate": 0.00018579834132349772,
227
+ "loss": 1.2422,
228
  "step": 140
229
  },
230
  {
231
+ "epoch": 2.6363636363636362,
232
+ "grad_norm": 0.62890625,
233
+ "learning_rate": 0.00018412535328311814,
234
+ "loss": 1.2256,
235
  "step": 145
236
  },
237
  {
238
+ "epoch": 2.7272727272727275,
239
+ "grad_norm": 2.421875,
240
+ "learning_rate": 0.0001823676581429833,
241
+ "loss": 1.2271,
242
  "step": 150
243
  },
244
  {
245
+ "epoch": 2.8181818181818183,
246
+ "grad_norm": 0.8515625,
247
+ "learning_rate": 0.00018052702575310588,
248
+ "loss": 1.2114,
249
  "step": 155
250
  },
251
  {
252
+ "epoch": 2.909090909090909,
253
+ "grad_norm": 0.56640625,
254
+ "learning_rate": 0.00017860530947427875,
255
+ "loss": 1.2068,
256
  "step": 160
257
  },
258
  {
259
+ "epoch": 3.0,
260
+ "grad_norm": 10.6875,
261
+ "learning_rate": 0.0001766044443118978,
262
+ "loss": 1.1944,
263
  "step": 165
264
  },
265
  {
266
+ "epoch": 3.0,
267
+ "eval_loss": 2.6026840209960938,
268
+ "eval_runtime": 0.2422,
269
+ "eval_samples_per_second": 41.281,
270
+ "eval_steps_per_second": 4.128,
271
+ "step": 165
272
+ },
273
+ {
274
+ "epoch": 3.090909090909091,
275
+ "grad_norm": 1.9140625,
276
+ "learning_rate": 0.0001745264449675755,
277
+ "loss": 1.194,
278
  "step": 170
279
  },
280
  {
281
+ "epoch": 3.1818181818181817,
282
+ "grad_norm": 3.515625,
283
+ "learning_rate": 0.00017237340381050703,
284
+ "loss": 1.1668,
285
  "step": 175
286
  },
287
  {
288
+ "epoch": 3.2727272727272725,
289
+ "grad_norm": 1.59375,
290
+ "learning_rate": 0.00017014748877063214,
291
+ "loss": 1.1693,
292
  "step": 180
293
  },
294
  {
295
+ "epoch": 3.3636363636363638,
296
+ "grad_norm": 2.109375,
297
+ "learning_rate": 0.00016785094115571322,
298
+ "loss": 1.167,
299
  "step": 185
300
  },
301
  {
302
+ "epoch": 3.4545454545454546,
303
+ "grad_norm": 1.7109375,
304
+ "learning_rate": 0.00016548607339452853,
305
+ "loss": 1.1601,
306
  "step": 190
307
  },
308
  {
309
+ "epoch": 3.5454545454545454,
310
+ "grad_norm": 1.0234375,
311
+ "learning_rate": 0.00016305526670845226,
312
+ "loss": 1.1343,
313
  "step": 195
314
  },
315
  {
316
+ "epoch": 3.6363636363636362,
317
+ "grad_norm": 1.265625,
318
+ "learning_rate": 0.00016056096871376667,
319
+ "loss": 1.1379,
320
  "step": 200
321
  },
322
  {
323
+ "epoch": 3.7272727272727275,
324
+ "grad_norm": 1.2109375,
325
+ "learning_rate": 0.00015800569095711982,
326
+ "loss": 1.1339,
 
 
 
 
 
 
 
 
327
  "step": 205
328
  },
329
  {
330
+ "epoch": 3.8181818181818183,
331
+ "grad_norm": 1.4296875,
332
+ "learning_rate": 0.00015539200638661104,
333
+ "loss": 1.1393,
334
  "step": 210
335
  },
336
  {
337
+ "epoch": 3.909090909090909,
338
+ "grad_norm": 0.7421875,
339
+ "learning_rate": 0.00015272254676105025,
340
+ "loss": 1.1015,
341
  "step": 215
342
  },
343
  {
344
+ "epoch": 4.0,
345
+ "grad_norm": 0.84765625,
346
+ "learning_rate": 0.00015000000000000001,
347
+ "loss": 1.1119,
348
  "step": 220
349
  },
350
  {
351
+ "epoch": 4.0,
352
+ "eval_loss": 2.5778679847717285,
353
+ "eval_runtime": 0.2336,
354
+ "eval_samples_per_second": 42.816,
355
+ "eval_steps_per_second": 4.282,
356
+ "step": 220
357
+ },
358
+ {
359
+ "epoch": 4.090909090909091,
360
+ "grad_norm": 1.5625,
361
+ "learning_rate": 0.0001472271074772683,
362
+ "loss": 1.113,
363
  "step": 225
364
  },
365
  {
366
+ "epoch": 4.181818181818182,
367
+ "grad_norm": 1.1171875,
368
+ "learning_rate": 0.00014440666126057744,
369
+ "loss": 1.0963,
370
  "step": 230
371
  },
372
  {
373
+ "epoch": 4.2727272727272725,
374
+ "grad_norm": 1.3359375,
375
+ "learning_rate": 0.00014154150130018866,
376
+ "loss": 1.104,
377
  "step": 235
378
  },
379
  {
380
+ "epoch": 4.363636363636363,
381
+ "grad_norm": 0.9296875,
382
+ "learning_rate": 0.00013863451256931287,
383
+ "loss": 1.0957,
384
  "step": 240
385
  },
386
  {
387
+ "epoch": 4.454545454545454,
388
+ "grad_norm": 1.9375,
389
+ "learning_rate": 0.00013568862215918717,
390
+ "loss": 1.0835,
391
  "step": 245
392
  },
393
  {
394
+ "epoch": 4.545454545454545,
395
+ "grad_norm": 1.1015625,
396
+ "learning_rate": 0.00013270679633174218,
397
+ "loss": 1.0802,
398
  "step": 250
399
  },
400
  {
401
+ "epoch": 4.636363636363637,
402
+ "grad_norm": 1.8515625,
403
+ "learning_rate": 0.0001296920375328275,
404
+ "loss": 1.0762,
405
  "step": 255
406
  },
407
  {
408
+ "epoch": 4.7272727272727275,
409
+ "grad_norm": 0.66796875,
410
+ "learning_rate": 0.00012664738136900348,
411
+ "loss": 1.0773,
412
  "step": 260
413
  },
414
  {
415
+ "epoch": 4.818181818181818,
416
+ "grad_norm": 0.80078125,
417
+ "learning_rate": 0.00012357589355094275,
418
+ "loss": 1.082,
419
  "step": 265
420
  },
421
  {
422
+ "epoch": 4.909090909090909,
423
+ "grad_norm": 1.7578125,
424
+ "learning_rate": 0.00012048066680651908,
425
+ "loss": 1.0687,
426
  "step": 270
427
  },
428
  {
429
+ "epoch": 5.0,
430
+ "grad_norm": 1.6484375,
431
+ "learning_rate": 0.00011736481776669306,
432
+ "loss": 1.0741,
433
  "step": 275
434
  },
435
  {
436
+ "epoch": 5.0,
437
+ "eval_loss": 2.5495071411132812,
438
+ "eval_runtime": 0.2341,
439
+ "eval_samples_per_second": 42.713,
440
+ "eval_steps_per_second": 4.271,
441
+ "step": 275
442
+ },
443
+ {
444
+ "epoch": 5.090909090909091,
445
+ "grad_norm": 0.79296875,
446
+ "learning_rate": 0.00011423148382732853,
447
+ "loss": 1.0546,
448
  "step": 280
449
  },
450
  {
451
+ "epoch": 5.181818181818182,
452
+ "grad_norm": 0.6484375,
453
+ "learning_rate": 0.00011108381999010111,
454
+ "loss": 1.0566,
455
  "step": 285
456
  },
457
  {
458
+ "epoch": 5.2727272727272725,
459
+ "grad_norm": 0.8984375,
460
+ "learning_rate": 0.00010792499568567884,
461
+ "loss": 1.0477,
462
  "step": 290
463
  },
464
  {
465
+ "epoch": 5.363636363636363,
466
+ "grad_norm": 0.890625,
467
+ "learning_rate": 0.00010475819158237425,
468
+ "loss": 1.0483,
469
  "step": 295
470
  },
471
  {
472
+ "epoch": 5.454545454545454,
473
+ "grad_norm": 0.546875,
474
+ "learning_rate": 0.00010158659638348081,
475
+ "loss": 1.0542,
476
  "step": 300
477
  },
478
  {
479
+ "epoch": 5.545454545454545,
480
+ "grad_norm": 0.57421875,
481
+ "learning_rate": 9.84134036165192e-05,
482
+ "loss": 1.0437,
483
  "step": 305
484
  },
485
  {
486
+ "epoch": 5.636363636363637,
487
+ "grad_norm": 0.76953125,
488
+ "learning_rate": 9.524180841762577e-05,
489
+ "loss": 1.0542,
490
  "step": 310
491
  },
492
  {
493
+ "epoch": 5.7272727272727275,
494
+ "grad_norm": 0.546875,
495
+ "learning_rate": 9.207500431432115e-05,
496
+ "loss": 1.0424,
497
  "step": 315
498
  },
499
  {
500
+ "epoch": 5.818181818181818,
501
+ "grad_norm": 0.494140625,
502
+ "learning_rate": 8.891618000989891e-05,
503
+ "loss": 1.0356,
504
  "step": 320
505
  },
506
  {
507
+ "epoch": 5.909090909090909,
508
+ "grad_norm": 0.58203125,
509
+ "learning_rate": 8.57685161726715e-05,
510
+ "loss": 1.0432,
511
  "step": 325
512
  },
513
  {
514
+ "epoch": 6.0,
515
+ "grad_norm": 0.55078125,
516
+ "learning_rate": 8.263518223330697e-05,
517
+ "loss": 1.0435,
518
+ "step": 330
519
+ },
520
+ {
521
+ "epoch": 6.0,
522
+ "eval_loss": 2.5500569343566895,
523
+ "eval_runtime": 0.2352,
524
+ "eval_samples_per_second": 42.524,
525
+ "eval_steps_per_second": 4.252,
526
  "step": 330
527
  },
528
  {
529
+ "epoch": 6.090909090909091,
530
+ "grad_norm": 0.59375,
531
+ "learning_rate": 7.951933319348095e-05,
532
+ "loss": 1.0185,
533
  "step": 335
534
  },
535
  {
536
+ "epoch": 6.181818181818182,
537
+ "grad_norm": 0.58203125,
538
+ "learning_rate": 7.642410644905726e-05,
539
+ "loss": 1.0249,
540
  "step": 340
541
  },
542
  {
543
+ "epoch": 6.2727272727272725,
544
+ "grad_norm": 0.6171875,
545
+ "learning_rate": 7.335261863099651e-05,
546
+ "loss": 1.0206,
547
  "step": 345
548
  },
549
  {
550
+ "epoch": 6.363636363636363,
551
+ "grad_norm": 0.6015625,
552
+ "learning_rate": 7.030796246717255e-05,
553
+ "loss": 1.0209,
554
  "step": 350
555
  },
556
  {
557
+ "epoch": 6.454545454545454,
558
+ "grad_norm": 0.5859375,
559
+ "learning_rate": 6.729320366825784e-05,
560
+ "loss": 1.0285,
561
  "step": 355
562
  },
563
  {
564
+ "epoch": 6.545454545454545,
565
+ "grad_norm": 0.75,
566
+ "learning_rate": 6.431137784081282e-05,
567
+ "loss": 1.0206,
568
  "step": 360
569
  },
570
  {
571
+ "epoch": 6.636363636363637,
572
+ "grad_norm": 0.5859375,
573
+ "learning_rate": 6.136548743068713e-05,
574
+ "loss": 1.0245,
575
  "step": 365
576
  },
577
  {
578
+ "epoch": 6.7272727272727275,
579
+ "grad_norm": 0.5078125,
580
+ "learning_rate": 5.845849869981137e-05,
581
+ "loss": 1.0174,
582
  "step": 370
583
  },
584
  {
585
+ "epoch": 6.818181818181818,
586
+ "grad_norm": 0.75,
587
+ "learning_rate": 5.559333873942259e-05,
588
+ "loss": 1.0206,
589
  "step": 375
590
  },
591
  {
592
+ "epoch": 6.909090909090909,
593
+ "grad_norm": 0.828125,
594
+ "learning_rate": 5.277289252273174e-05,
595
+ "loss": 1.0244,
596
  "step": 380
597
  },
598
  {
599
+ "epoch": 7.0,
600
+ "grad_norm": 0.83203125,
601
+ "learning_rate": 5.000000000000002e-05,
602
+ "loss": 1.0191,
603
+ "step": 385
604
+ },
605
+ {
606
+ "epoch": 7.0,
607
+ "eval_loss": 2.5536184310913086,
608
+ "eval_runtime": 0.2341,
609
+ "eval_samples_per_second": 42.722,
610
+ "eval_steps_per_second": 4.272,
611
  "step": 385
612
  },
613
  {
614
+ "epoch": 7.090909090909091,
615
+ "grad_norm": 0.7734375,
616
+ "learning_rate": 4.727745323894976e-05,
617
+ "loss": 1.011,
618
  "step": 390
619
  },
620
  {
621
+ "epoch": 7.181818181818182,
622
+ "grad_norm": 0.61328125,
623
+ "learning_rate": 4.4607993613388976e-05,
624
+ "loss": 1.0087,
625
  "step": 395
626
  },
627
  {
628
+ "epoch": 7.2727272727272725,
629
+ "grad_norm": 0.6875,
630
+ "learning_rate": 4.19943090428802e-05,
631
+ "loss": 1.0019,
632
  "step": 400
633
  },
634
  {
635
+ "epoch": 7.363636363636363,
636
+ "grad_norm": 0.7890625,
637
+ "learning_rate": 3.943903128623335e-05,
638
+ "loss": 1.0112,
639
  "step": 405
640
  },
641
  {
642
+ "epoch": 7.454545454545454,
643
+ "grad_norm": 0.58203125,
644
+ "learning_rate": 3.694473329154778e-05,
645
+ "loss": 1.0075,
 
 
 
 
 
 
 
 
646
  "step": 410
647
  },
648
  {
649
+ "epoch": 7.545454545454545,
650
+ "grad_norm": 0.5703125,
651
+ "learning_rate": 3.45139266054715e-05,
652
+ "loss": 1.0023,
653
  "step": 415
654
  },
655
  {
656
+ "epoch": 7.636363636363637,
657
+ "grad_norm": 0.8046875,
658
+ "learning_rate": 3.21490588442868e-05,
659
+ "loss": 1.0112,
660
  "step": 420
661
  },
662
  {
663
+ "epoch": 7.7272727272727275,
664
+ "grad_norm": 0.4765625,
665
+ "learning_rate": 2.9852511229367865e-05,
666
+ "loss": 1.0069,
667
  "step": 425
668
  },
669
  {
670
+ "epoch": 7.818181818181818,
671
+ "grad_norm": 0.45703125,
672
+ "learning_rate": 2.7626596189492983e-05,
673
+ "loss": 1.002,
674
  "step": 430
675
  },
676
  {
677
+ "epoch": 7.909090909090909,
678
+ "grad_norm": 0.55078125,
679
+ "learning_rate": 2.5473555032424533e-05,
680
+ "loss": 1.0076,
681
  "step": 435
682
  },
683
  {
684
+ "epoch": 8.0,
685
+ "grad_norm": 0.478515625,
686
+ "learning_rate": 2.339555568810221e-05,
687
+ "loss": 0.9965,
688
+ "step": 440
689
+ },
690
+ {
691
+ "epoch": 8.0,
692
+ "eval_loss": 2.5604465007781982,
693
+ "eval_runtime": 0.24,
694
+ "eval_samples_per_second": 41.665,
695
+ "eval_steps_per_second": 4.167,
696
  "step": 440
697
  },
698
  {
699
+ "epoch": 8.090909090909092,
700
+ "grad_norm": 0.61328125,
701
+ "learning_rate": 2.139469052572127e-05,
702
+ "loss": 1.0043,
703
  "step": 445
704
  },
705
  {
706
+ "epoch": 8.181818181818182,
707
+ "grad_norm": 0.58984375,
708
+ "learning_rate": 1.947297424689414e-05,
709
+ "loss": 0.9997,
710
  "step": 450
711
  },
712
  {
713
+ "epoch": 8.272727272727273,
714
+ "grad_norm": 0.57421875,
715
+ "learning_rate": 1.763234185701673e-05,
716
+ "loss": 0.9942,
717
  "step": 455
718
  },
719
  {
720
+ "epoch": 8.363636363636363,
721
+ "grad_norm": 0.5625,
722
+ "learning_rate": 1.587464671688187e-05,
723
+ "loss": 0.9952,
724
  "step": 460
725
  },
726
  {
727
+ "epoch": 8.454545454545455,
728
+ "grad_norm": 0.453125,
729
+ "learning_rate": 1.4201658676502294e-05,
730
+ "loss": 1.0037,
731
  "step": 465
732
  },
733
  {
734
+ "epoch": 8.545454545454545,
735
+ "grad_norm": 0.419921875,
736
+ "learning_rate": 1.2615062293021507e-05,
737
+ "loss": 1.0051,
738
  "step": 470
739
  },
740
  {
741
+ "epoch": 8.636363636363637,
742
+ "grad_norm": 0.484375,
743
+ "learning_rate": 1.1116455134507664e-05,
744
+ "loss": 0.9905,
745
  "step": 475
746
  },
747
  {
748
+ "epoch": 8.727272727272727,
749
+ "grad_norm": 0.44140625,
750
+ "learning_rate": 9.707346171337894e-06,
751
+ "loss": 0.9977,
752
  "step": 480
753
  },
754
  {
755
+ "epoch": 8.818181818181818,
756
+ "grad_norm": 0.43359375,
757
+ "learning_rate": 8.38915425679304e-06,
758
+ "loss": 0.9894,
759
  "step": 485
760
  },
761
  {
762
+ "epoch": 8.909090909090908,
763
+ "grad_norm": 0.470703125,
764
+ "learning_rate": 7.163206698392744e-06,
765
+ "loss": 1.005,
766
  "step": 490
767
  },
768
  {
769
+ "epoch": 9.0,
770
+ "grad_norm": 0.42578125,
771
+ "learning_rate": 6.030737921409169e-06,
772
+ "loss": 0.9986,
773
+ "step": 495
774
+ },
775
+ {
776
+ "epoch": 9.0,
777
+ "eval_loss": 2.5596535205841064,
778
+ "eval_runtime": 0.2334,
779
+ "eval_samples_per_second": 42.846,
780
+ "eval_steps_per_second": 4.285,
781
  "step": 495
782
  },
783
  {
784
+ "epoch": 9.090909090909092,
785
+ "grad_norm": 0.455078125,
786
+ "learning_rate": 4.992888225905468e-06,
787
+ "loss": 0.9957,
788
  "step": 500
789
  },
790
  {
791
+ "epoch": 9.181818181818182,
792
+ "grad_norm": 0.47265625,
793
+ "learning_rate": 4.050702638550275e-06,
794
+ "loss": 1.0036,
795
  "step": 505
796
  },
797
  {
798
+ "epoch": 9.272727272727273,
799
+ "grad_norm": 0.462890625,
800
+ "learning_rate": 3.2051298603643753e-06,
801
+ "loss": 0.9985,
802
  "step": 510
803
  },
804
  {
805
+ "epoch": 9.363636363636363,
806
+ "grad_norm": 0.458984375,
807
+ "learning_rate": 2.4570213114592954e-06,
808
+ "loss": 0.9961,
809
  "step": 515
810
  },
811
  {
812
+ "epoch": 9.454545454545455,
813
+ "grad_norm": 0.443359375,
814
+ "learning_rate": 1.8071302737293295e-06,
815
+ "loss": 1.0066,
816
  "step": 520
817
  },
818
  {
819
+ "epoch": 9.545454545454545,
820
+ "grad_norm": 0.43359375,
821
+ "learning_rate": 1.2561111323605712e-06,
822
+ "loss": 0.9996,
823
  "step": 525
824
  },
825
  {
826
+ "epoch": 9.636363636363637,
827
+ "grad_norm": 0.439453125,
828
+ "learning_rate": 8.04518716920466e-07,
829
+ "loss": 0.9941,
830
  "step": 530
831
  },
832
  {
833
+ "epoch": 9.727272727272727,
834
+ "grad_norm": 0.419921875,
835
+ "learning_rate": 4.5280774269154115e-07,
836
+ "loss": 0.9898,
837
  "step": 535
838
  },
839
  {
840
+ "epoch": 9.818181818181818,
841
+ "grad_norm": 0.419921875,
842
+ "learning_rate": 2.0133235281156736e-07,
843
+ "loss": 0.9888,
844
  "step": 540
845
  },
846
  {
847
+ "epoch": 9.909090909090908,
848
+ "grad_norm": 0.4453125,
849
+ "learning_rate": 5.0345761681491746e-08,
850
+ "loss": 0.9953,
851
  "step": 545
852
  },
853
  {
854
+ "epoch": 10.0,
855
+ "grad_norm": 0.41796875,
856
+ "learning_rate": 0.0,
857
+ "loss": 0.9948,
858
  "step": 550
859
  },
860
  {
861
+ "epoch": 10.0,
862
+ "eval_loss": 2.5602283477783203,
863
+ "eval_runtime": 0.2428,
864
+ "eval_samples_per_second": 41.182,
865
+ "eval_steps_per_second": 4.118,
866
+ "step": 550
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
867
  },
868
  {
869
+ "epoch": 10.0,
870
+ "step": 550,
871
+ "total_flos": 1.6777423328808796e+18,
872
+ "train_loss": 3.202145513187755,
873
+ "train_runtime": 1331.6624,
874
+ "train_samples_per_second": 26.313,
875
+ "train_steps_per_second": 0.413
876
  }
877
  ],
878
  "logging_steps": 5,
879
+ "max_steps": 550,
880
  "num_input_tokens_seen": 0,
881
  "num_train_epochs": 10,
882
  "save_steps": 100,
883
+ "total_flos": 1.6777423328808796e+18,
884
  "train_batch_size": 4,
885
  "trial_name": null,
886
  "trial_params": null
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:281bac798d4f46cabe9ee13145f7c00c9bfe2797573fcfc2ff4225f62bc6a512
3
  size 5176
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd097c126007f600ecfb0e51962f9ceca5bf741f3de46df16b3ca140da5142dc
3
  size 5176