Hugging Face's logo

akahana
/
tinybert-javanese

Fill-Mask
Transformers
TensorBoard
Safetensors
bert
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Metrics Community
akahana commited on
Commit
34bd5ab
•
1 Parent(s): e9ab5ca

End of training

Browse files
Files changed (6) hide show
  1. README.md +20 -2
  2. all_results.json +16 -0
  3. eval_results.json +10 -0
  4. runs/Jul30_00-37-31_8ff5e8032af8/events.out.tfevents.1722307823.8ff5e8032af8.1173.1 +3 -0
  5. train_results.json +9 -0
  6. trainer_state.json +3381 -0
README.md CHANGED
@@ -1,9 +1,24 @@
1
  ---
2
  tags:
3
  - generated_from_trainer
 
 
 
 
4
  model-index:
5
  - name: tinybert-javanese
6
- results: []
 
 
 
 
 
 
 
 
 
 
 
7
  ---
8
 
9
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -11,7 +26,10 @@ should probably proofread and complete it, then remove this comment. -->
11
 
12
  # tinybert-javanese
13
 
14
- This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 
 
 
15
 
16
  ## Model description
17
 
 
1
  ---
2
  tags:
3
  - generated_from_trainer
4
+ datasets:
5
+ - akahana/GlotCC-V1-jav-Latn
6
+ metrics:
7
+ - accuracy
8
  model-index:
9
  - name: tinybert-javanese
10
+ results:
11
+ - task:
12
+ name: Masked Language Modeling
13
+ type: fill-mask
14
+ dataset:
15
+ name: akahana/GlotCC-V1-jav-Latn default
16
+ type: akahana/GlotCC-V1-jav-Latn
17
+ args: default
18
+ metrics:
19
+ - name: Accuracy
20
+ type: accuracy
21
+ value: 0.1519208618470086
22
  ---
23
 
24
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
26
 
27
  # tinybert-javanese
28
 
29
+ This model is a fine-tuned version of [](https://huggingface.co/) on the akahana/GlotCC-V1-jav-Latn default dataset.
30
+ It achieves the following results on the evaluation set:
31
+ - Loss: 5.8252
32
+ - Accuracy: 0.1519
33
 
34
  ## Model description
35
 
all_results.json ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 50.0,
3
+ "eval_accuracy": 0.1519208618470086,
4
+ "eval_loss": 5.8251872062683105,
5
+ "eval_runtime": 11.1208,
6
+ "eval_samples": 3852,
7
+ "eval_samples_per_second": 346.376,
8
+ "eval_steps_per_second": 86.594,
9
+ "perplexity": 338.7245384996336,
10
+ "total_flos": 5161725447936000.0,
11
+ "train_loss": 6.140905278960581,
12
+ "train_runtime": 7883.0646,
13
+ "train_samples": 76378,
14
+ "train_samples_per_second": 484.444,
15
+ "train_steps_per_second": 30.28
16
+ }
eval_results.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 50.0,
3
+ "eval_accuracy": 0.1519208618470086,
4
+ "eval_loss": 5.8251872062683105,
5
+ "eval_runtime": 11.1208,
6
+ "eval_samples": 3852,
7
+ "eval_samples_per_second": 346.376,
8
+ "eval_steps_per_second": 86.594,
9
+ "perplexity": 338.7245384996336
10
+ }
runs/Jul30_00-37-31_8ff5e8032af8/events.out.tfevents.1722307823.8ff5e8032af8.1173.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6fd5cd4ca647ecdbbc92d2c3fd74bd22d3bb62df7be93d03a8546aa5a3ff644
3
+ size 417
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 50.0,
3
+ "total_flos": 5161725447936000.0,
4
+ "train_loss": 6.140905278960581,
5
+ "train_runtime": 7883.0646,
6
+ "train_samples": 76378,
7
+ "train_samples_per_second": 484.444,
8
+ "train_steps_per_second": 30.28
9
+ }
trainer_state.json ADDED
@@ -0,0 +1,3381 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 50.0,
5
+ "eval_steps": 500,
6
+ "global_step": 238700,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.10473397570171764,
13
+ "grad_norm": 1.1384308338165283,
14
+ "learning_rate": 4.989526602429829e-05,
15
+ "loss": 8.9672,
16
+ "step": 500
17
+ },
18
+ {
19
+ "epoch": 0.20946795140343527,
20
+ "grad_norm": 0.9985808730125427,
21
+ "learning_rate": 4.979053204859657e-05,
22
+ "loss": 7.7253,
23
+ "step": 1000
24
+ },
25
+ {
26
+ "epoch": 0.31420192710515293,
27
+ "grad_norm": 1.0521348714828491,
28
+ "learning_rate": 4.968579807289485e-05,
29
+ "loss": 7.5614,
30
+ "step": 1500
31
+ },
32
+ {
33
+ "epoch": 0.41893590280687054,
34
+ "grad_norm": 1.0602227449417114,
35
+ "learning_rate": 4.958106409719313e-05,
36
+ "loss": 7.5037,
37
+ "step": 2000
38
+ },
39
+ {
40
+ "epoch": 0.5236698785085881,
41
+ "grad_norm": 1.6102268695831299,
42
+ "learning_rate": 4.9476330121491414e-05,
43
+ "loss": 7.4595,
44
+ "step": 2500
45
+ },
46
+ {
47
+ "epoch": 0.6284038542103059,
48
+ "grad_norm": 1.335976004600525,
49
+ "learning_rate": 4.9371596145789694e-05,
50
+ "loss": 7.4267,
51
+ "step": 3000
52
+ },
53
+ {
54
+ "epoch": 0.7331378299120235,
55
+ "grad_norm": 1.340728998184204,
56
+ "learning_rate": 4.926686217008798e-05,
57
+ "loss": 7.392,
58
+ "step": 3500
59
+ },
60
+ {
61
+ "epoch": 0.8378718056137411,
62
+ "grad_norm": 1.4520059823989868,
63
+ "learning_rate": 4.916212819438626e-05,
64
+ "loss": 7.3253,
65
+ "step": 4000
66
+ },
67
+ {
68
+ "epoch": 0.9426057813154587,
69
+ "grad_norm": 1.7685532569885254,
70
+ "learning_rate": 4.905760368663595e-05,
71
+ "loss": 7.3204,
72
+ "step": 4500
73
+ },
74
+ {
75
+ "epoch": 1.0473397570171763,
76
+ "grad_norm": 1.396130084991455,
77
+ "learning_rate": 4.8952869710934226e-05,
78
+ "loss": 7.2682,
79
+ "step": 5000
80
+ },
81
+ {
82
+ "epoch": 1.1520737327188941,
83
+ "grad_norm": 1.5962079763412476,
84
+ "learning_rate": 4.884813573523251e-05,
85
+ "loss": 7.2441,
86
+ "step": 5500
87
+ },
88
+ {
89
+ "epoch": 1.2568077084206117,
90
+ "grad_norm": 1.6328166723251343,
91
+ "learning_rate": 4.874340175953079e-05,
92
+ "loss": 7.2216,
93
+ "step": 6000
94
+ },
95
+ {
96
+ "epoch": 1.3615416841223293,
97
+ "grad_norm": 1.8534362316131592,
98
+ "learning_rate": 4.863887725178048e-05,
99
+ "loss": 7.1957,
100
+ "step": 6500
101
+ },
102
+ {
103
+ "epoch": 1.466275659824047,
104
+ "grad_norm": 1.4871692657470703,
105
+ "learning_rate": 4.8534143276078766e-05,
106
+ "loss": 7.1561,
107
+ "step": 7000
108
+ },
109
+ {
110
+ "epoch": 1.5710096355257646,
111
+ "grad_norm": 1.8590672016143799,
112
+ "learning_rate": 4.8429409300377045e-05,
113
+ "loss": 7.1397,
114
+ "step": 7500
115
+ },
116
+ {
117
+ "epoch": 1.6757436112274822,
118
+ "grad_norm": 1.7009446620941162,
119
+ "learning_rate": 4.8324675324675325e-05,
120
+ "loss": 7.1149,
121
+ "step": 8000
122
+ },
123
+ {
124
+ "epoch": 1.7804775869291998,
125
+ "grad_norm": 1.9020010232925415,
126
+ "learning_rate": 4.822015081692501e-05,
127
+ "loss": 7.1215,
128
+ "step": 8500
129
+ },
130
+ {
131
+ "epoch": 1.8852115626309174,
132
+ "grad_norm": 2.912442445755005,
133
+ "learning_rate": 4.811541684122329e-05,
134
+ "loss": 7.0616,
135
+ "step": 9000
136
+ },
137
+ {
138
+ "epoch": 1.989945538332635,
139
+ "grad_norm": 2.654263496398926,
140
+ "learning_rate": 4.801068286552158e-05,
141
+ "loss": 7.0508,
142
+ "step": 9500
143
+ },
144
+ {
145
+ "epoch": 2.0946795140343526,
146
+ "grad_norm": 2.1642003059387207,
147
+ "learning_rate": 4.790594888981986e-05,
148
+ "loss": 7.0251,
149
+ "step": 10000
150
+ },
151
+ {
152
+ "epoch": 2.19941348973607,
153
+ "grad_norm": 1.9420874118804932,
154
+ "learning_rate": 4.7801424382069544e-05,
155
+ "loss": 6.9806,
156
+ "step": 10500
157
+ },
158
+ {
159
+ "epoch": 2.3041474654377883,
160
+ "grad_norm": 2.2306201457977295,
161
+ "learning_rate": 4.769669040636783e-05,
162
+ "loss": 6.9721,
163
+ "step": 11000
164
+ },
165
+ {
166
+ "epoch": 2.4088814411395054,
167
+ "grad_norm": 2.8180389404296875,
168
+ "learning_rate": 4.759195643066611e-05,
169
+ "loss": 6.9582,
170
+ "step": 11500
171
+ },
172
+ {
173
+ "epoch": 2.5136154168412235,
174
+ "grad_norm": 2.387949228286743,
175
+ "learning_rate": 4.748722245496439e-05,
176
+ "loss": 6.9211,
177
+ "step": 12000
178
+ },
179
+ {
180
+ "epoch": 2.618349392542941,
181
+ "grad_norm": 3.3709394931793213,
182
+ "learning_rate": 4.7382697947214076e-05,
183
+ "loss": 6.9183,
184
+ "step": 12500
185
+ },
186
+ {
187
+ "epoch": 2.7230833682446587,
188
+ "grad_norm": 2.567798376083374,
189
+ "learning_rate": 4.727796397151236e-05,
190
+ "loss": 6.8732,
191
+ "step": 13000
192
+ },
193
+ {
194
+ "epoch": 2.8278173439463763,
195
+ "grad_norm": 2.6373414993286133,
196
+ "learning_rate": 4.717322999581064e-05,
197
+ "loss": 6.8658,
198
+ "step": 13500
199
+ },
200
+ {
201
+ "epoch": 2.932551319648094,
202
+ "grad_norm": 2.2950875759124756,
203
+ "learning_rate": 4.706849602010893e-05,
204
+ "loss": 6.8436,
205
+ "step": 14000
206
+ },
207
+ {
208
+ "epoch": 3.0372852953498115,
209
+ "grad_norm": 4.0021514892578125,
210
+ "learning_rate": 4.696397151235861e-05,
211
+ "loss": 6.8496,
212
+ "step": 14500
213
+ },
214
+ {
215
+ "epoch": 3.142019271051529,
216
+ "grad_norm": 3.289193630218506,
217
+ "learning_rate": 4.685923753665689e-05,
218
+ "loss": 6.8047,
219
+ "step": 15000
220
+ },
221
+ {
222
+ "epoch": 3.2467532467532467,
223
+ "grad_norm": 2.9973654747009277,
224
+ "learning_rate": 4.6754503560955175e-05,
225
+ "loss": 6.7651,
226
+ "step": 15500
227
+ },
228
+ {
229
+ "epoch": 3.3514872224549643,
230
+ "grad_norm": 2.9979376792907715,
231
+ "learning_rate": 4.664976958525346e-05,
232
+ "loss": 6.7768,
233
+ "step": 16000
234
+ },
235
+ {
236
+ "epoch": 3.456221198156682,
237
+ "grad_norm": 3.263784885406494,
238
+ "learning_rate": 4.654524507750315e-05,
239
+ "loss": 6.7617,
240
+ "step": 16500
241
+ },
242
+ {
243
+ "epoch": 3.5609551738583995,
244
+ "grad_norm": 3.330116033554077,
245
+ "learning_rate": 4.644051110180143e-05,
246
+ "loss": 6.7417,
247
+ "step": 17000
248
+ },
249
+ {
250
+ "epoch": 3.665689149560117,
251
+ "grad_norm": 3.224337339401245,
252
+ "learning_rate": 4.6335777126099714e-05,
253
+ "loss": 6.7028,
254
+ "step": 17500
255
+ },
256
+ {
257
+ "epoch": 3.7704231252618348,
258
+ "grad_norm": 3.21891450881958,
259
+ "learning_rate": 4.623104315039799e-05,
260
+ "loss": 6.7101,
261
+ "step": 18000
262
+ },
263
+ {
264
+ "epoch": 3.875157100963553,
265
+ "grad_norm": 2.3559324741363525,
266
+ "learning_rate": 4.6126518642647674e-05,
267
+ "loss": 6.6892,
268
+ "step": 18500
269
+ },
270
+ {
271
+ "epoch": 3.97989107666527,
272
+ "grad_norm": 3.1527633666992188,
273
+ "learning_rate": 4.602178466694596e-05,
274
+ "loss": 6.6851,
275
+ "step": 19000
276
+ },
277
+ {
278
+ "epoch": 4.084625052366988,
279
+ "grad_norm": 2.9760189056396484,
280
+ "learning_rate": 4.591705069124424e-05,
281
+ "loss": 6.6704,
282
+ "step": 19500
283
+ },
284
+ {
285
+ "epoch": 4.189359028068705,
286
+ "grad_norm": 2.8135318756103516,
287
+ "learning_rate": 4.5812316715542526e-05,
288
+ "loss": 6.6621,
289
+ "step": 20000
290
+ },
291
+ {
292
+ "epoch": 4.294093003770423,
293
+ "grad_norm": 3.060316324234009,
294
+ "learning_rate": 4.570779220779221e-05,
295
+ "loss": 6.6298,
296
+ "step": 20500
297
+ },
298
+ {
299
+ "epoch": 4.39882697947214,
300
+ "grad_norm": 2.7130279541015625,
301
+ "learning_rate": 4.560305823209049e-05,
302
+ "loss": 6.628,
303
+ "step": 21000
304
+ },
305
+ {
306
+ "epoch": 4.5035609551738585,
307
+ "grad_norm": 3.156386613845825,
308
+ "learning_rate": 4.549853372434017e-05,
309
+ "loss": 6.6423,
310
+ "step": 21500
311
+ },
312
+ {
313
+ "epoch": 4.6082949308755765,
314
+ "grad_norm": 3.039471387863159,
315
+ "learning_rate": 4.539379974863846e-05,
316
+ "loss": 6.6343,
317
+ "step": 22000
318
+ },
319
+ {
320
+ "epoch": 4.713028906577294,
321
+ "grad_norm": 3.976949453353882,
322
+ "learning_rate": 4.5289065772936745e-05,
323
+ "loss": 6.606,
324
+ "step": 22500
325
+ },
326
+ {
327
+ "epoch": 4.817762882279011,
328
+ "grad_norm": 3.310382604598999,
329
+ "learning_rate": 4.5184331797235025e-05,
330
+ "loss": 6.5956,
331
+ "step": 23000
332
+ },
333
+ {
334
+ "epoch": 4.922496857980729,
335
+ "grad_norm": 3.5924322605133057,
336
+ "learning_rate": 4.507959782153331e-05,
337
+ "loss": 6.5965,
338
+ "step": 23500
339
+ },
340
+ {
341
+ "epoch": 5.027230833682447,
342
+ "grad_norm": 2.616468667984009,
343
+ "learning_rate": 4.497486384583159e-05,
344
+ "loss": 6.5788,
345
+ "step": 24000
346
+ },
347
+ {
348
+ "epoch": 5.131964809384164,
349
+ "grad_norm": 3.3178062438964844,
350
+ "learning_rate": 4.487012987012987e-05,
351
+ "loss": 6.5684,
352
+ "step": 24500
353
+ },
354
+ {
355
+ "epoch": 5.236698785085882,
356
+ "grad_norm": 3.7108089923858643,
357
+ "learning_rate": 4.476539589442815e-05,
358
+ "loss": 6.5756,
359
+ "step": 25000
360
+ },
361
+ {
362
+ "epoch": 5.341432760787599,
363
+ "grad_norm": 3.396498918533325,
364
+ "learning_rate": 4.466087138667784e-05,
365
+ "loss": 6.5678,
366
+ "step": 25500
367
+ },
368
+ {
369
+ "epoch": 5.446166736489317,
370
+ "grad_norm": 3.7245748043060303,
371
+ "learning_rate": 4.4556137410976123e-05,
372
+ "loss": 6.5578,
373
+ "step": 26000
374
+ },
375
+ {
376
+ "epoch": 5.5509007121910345,
377
+ "grad_norm": 3.6525135040283203,
378
+ "learning_rate": 4.44514034352744e-05,
379
+ "loss": 6.5385,
380
+ "step": 26500
381
+ },
382
+ {
383
+ "epoch": 5.655634687892753,
384
+ "grad_norm": 3.4302523136138916,
385
+ "learning_rate": 4.434666945957269e-05,
386
+ "loss": 6.5143,
387
+ "step": 27000
388
+ },
389
+ {
390
+ "epoch": 5.76036866359447,
391
+ "grad_norm": 3.762871503829956,
392
+ "learning_rate": 4.4242144951822376e-05,
393
+ "loss": 6.52,
394
+ "step": 27500
395
+ },
396
+ {
397
+ "epoch": 5.865102639296188,
398
+ "grad_norm": 2.8195388317108154,
399
+ "learning_rate": 4.4137410976120656e-05,
400
+ "loss": 6.5213,
401
+ "step": 28000
402
+ },
403
+ {
404
+ "epoch": 5.969836614997905,
405
+ "grad_norm": 3.013187885284424,
406
+ "learning_rate": 4.4032677000418936e-05,
407
+ "loss": 6.5052,
408
+ "step": 28500
409
+ },
410
+ {
411
+ "epoch": 6.074570590699623,
412
+ "grad_norm": 2.9772274494171143,
413
+ "learning_rate": 4.392794302471722e-05,
414
+ "loss": 6.502,
415
+ "step": 29000
416
+ },
417
+ {
418
+ "epoch": 6.17930456640134,
419
+ "grad_norm": 3.2228713035583496,
420
+ "learning_rate": 4.38232090490155e-05,
421
+ "loss": 6.4889,
422
+ "step": 29500
423
+ },
424
+ {
425
+ "epoch": 6.284038542103058,
426
+ "grad_norm": 3.824286937713623,
427
+ "learning_rate": 4.371847507331379e-05,
428
+ "loss": 6.4792,
429
+ "step": 30000
430
+ },
431
+ {
432
+ "epoch": 6.388772517804776,
433
+ "grad_norm": 3.100308656692505,
434
+ "learning_rate": 4.361374109761207e-05,
435
+ "loss": 6.4816,
436
+ "step": 30500
437
+ },
438
+ {
439
+ "epoch": 6.4935064935064934,
440
+ "grad_norm": 3.4449245929718018,
441
+ "learning_rate": 4.350900712191035e-05,
442
+ "loss": 6.4786,
443
+ "step": 31000
444
+ },
445
+ {
446
+ "epoch": 6.5982404692082115,
447
+ "grad_norm": 3.6803085803985596,
448
+ "learning_rate": 4.3404482614160034e-05,
449
+ "loss": 6.4778,
450
+ "step": 31500
451
+ },
452
+ {
453
+ "epoch": 6.702974444909929,
454
+ "grad_norm": 3.6413722038269043,
455
+ "learning_rate": 4.329974863845832e-05,
456
+ "loss": 6.4782,
457
+ "step": 32000
458
+ },
459
+ {
460
+ "epoch": 6.807708420611647,
461
+ "grad_norm": 3.482905626296997,
462
+ "learning_rate": 4.31950146627566e-05,
463
+ "loss": 6.4719,
464
+ "step": 32500
465
+ },
466
+ {
467
+ "epoch": 6.912442396313364,
468
+ "grad_norm": 3.4605376720428467,
469
+ "learning_rate": 4.3090280687054887e-05,
470
+ "loss": 6.4458,
471
+ "step": 33000
472
+ },
473
+ {
474
+ "epoch": 7.017176372015082,
475
+ "grad_norm": 3.814375877380371,
476
+ "learning_rate": 4.2985965647255974e-05,
477
+ "loss": 6.4539,
478
+ "step": 33500
479
+ },
480
+ {
481
+ "epoch": 7.121910347716799,
482
+ "grad_norm": 4.238844871520996,
483
+ "learning_rate": 4.288123167155425e-05,
484
+ "loss": 6.4522,
485
+ "step": 34000
486
+ },
487
+ {
488
+ "epoch": 7.226644323418517,
489
+ "grad_norm": 2.8327670097351074,
490
+ "learning_rate": 4.277649769585253e-05,
491
+ "loss": 6.4383,
492
+ "step": 34500
493
+ },
494
+ {
495
+ "epoch": 7.331378299120234,
496
+ "grad_norm": 3.1451475620269775,
497
+ "learning_rate": 4.267176372015082e-05,
498
+ "loss": 6.4451,
499
+ "step": 35000
500
+ },
501
+ {
502
+ "epoch": 7.436112274821952,
503
+ "grad_norm": 3.6858575344085693,
504
+ "learning_rate": 4.2567239212400506e-05,
505
+ "loss": 6.4313,
506
+ "step": 35500
507
+ },
508
+ {
509
+ "epoch": 7.5408462505236695,
510
+ "grad_norm": 4.258295059204102,
511
+ "learning_rate": 4.2462505236698786e-05,
512
+ "loss": 6.4143,
513
+ "step": 36000
514
+ },
515
+ {
516
+ "epoch": 7.645580226225388,
517
+ "grad_norm": 4.3574676513671875,
518
+ "learning_rate": 4.235777126099707e-05,
519
+ "loss": 6.4061,
520
+ "step": 36500
521
+ },
522
+ {
523
+ "epoch": 7.750314201927106,
524
+ "grad_norm": 3.8001816272735596,
525
+ "learning_rate": 4.225303728529535e-05,
526
+ "loss": 6.3994,
527
+ "step": 37000
528
+ },
529
+ {
530
+ "epoch": 7.855048177628823,
531
+ "grad_norm": 3.487893581390381,
532
+ "learning_rate": 4.214851277754504e-05,
533
+ "loss": 6.4192,
534
+ "step": 37500
535
+ },
536
+ {
537
+ "epoch": 7.95978215333054,
538
+ "grad_norm": 3.9729723930358887,
539
+ "learning_rate": 4.204377880184332e-05,
540
+ "loss": 6.407,
541
+ "step": 38000
542
+ },
543
+ {
544
+ "epoch": 8.064516129032258,
545
+ "grad_norm": 3.4465062618255615,
546
+ "learning_rate": 4.1939044826141604e-05,
547
+ "loss": 6.3867,
548
+ "step": 38500
549
+ },
550
+ {
551
+ "epoch": 8.169250104733976,
552
+ "grad_norm": 3.706404685974121,
553
+ "learning_rate": 4.1834310850439884e-05,
554
+ "loss": 6.3877,
555
+ "step": 39000
556
+ },
557
+ {
558
+ "epoch": 8.273984080435694,
559
+ "grad_norm": 3.8204259872436523,
560
+ "learning_rate": 4.172957687473817e-05,
561
+ "loss": 6.3921,
562
+ "step": 39500
563
+ },
564
+ {
565
+ "epoch": 8.37871805613741,
566
+ "grad_norm": 3.4868948459625244,
567
+ "learning_rate": 4.162505236698786e-05,
568
+ "loss": 6.3729,
569
+ "step": 40000
570
+ },
571
+ {
572
+ "epoch": 8.483452031839128,
573
+ "grad_norm": 3.5007081031799316,
574
+ "learning_rate": 4.152031839128614e-05,
575
+ "loss": 6.3886,
576
+ "step": 40500
577
+ },
578
+ {
579
+ "epoch": 8.588186007540846,
580
+ "grad_norm": 2.937894582748413,
581
+ "learning_rate": 4.1415584415584417e-05,
582
+ "loss": 6.3814,
583
+ "step": 41000
584
+ },
585
+ {
586
+ "epoch": 8.692919983242565,
587
+ "grad_norm": 3.529237985610962,
588
+ "learning_rate": 4.1310850439882696e-05,
589
+ "loss": 6.3722,
590
+ "step": 41500
591
+ },
592
+ {
593
+ "epoch": 8.79765395894428,
594
+ "grad_norm": 3.883575677871704,
595
+ "learning_rate": 4.120611646418098e-05,
596
+ "loss": 6.3655,
597
+ "step": 42000
598
+ },
599
+ {
600
+ "epoch": 8.902387934645999,
601
+ "grad_norm": 4.439103603363037,
602
+ "learning_rate": 4.110159195643067e-05,
603
+ "loss": 6.3673,
604
+ "step": 42500
605
+ },
606
+ {
607
+ "epoch": 9.007121910347717,
608
+ "grad_norm": 4.103298664093018,
609
+ "learning_rate": 4.099685798072895e-05,
610
+ "loss": 6.3659,
611
+ "step": 43000
612
+ },
613
+ {
614
+ "epoch": 9.111855886049435,
615
+ "grad_norm": 3.491204023361206,
616
+ "learning_rate": 4.0892124005027235e-05,
617
+ "loss": 6.3744,
618
+ "step": 43500
619
+ },
620
+ {
621
+ "epoch": 9.216589861751151,
622
+ "grad_norm": 3.441976547241211,
623
+ "learning_rate": 4.0787390029325515e-05,
624
+ "loss": 6.3573,
625
+ "step": 44000
626
+ },
627
+ {
628
+ "epoch": 9.32132383745287,
629
+ "grad_norm": 3.58134126663208,
630
+ "learning_rate": 4.0682656053623795e-05,
631
+ "loss": 6.3407,
632
+ "step": 44500
633
+ },
634
+ {
635
+ "epoch": 9.426057813154587,
636
+ "grad_norm": 3.274592638015747,
637
+ "learning_rate": 4.057813154587348e-05,
638
+ "loss": 6.3373,
639
+ "step": 45000
640
+ },
641
+ {
642
+ "epoch": 9.530791788856305,
643
+ "grad_norm": 4.296390533447266,
644
+ "learning_rate": 4.047339757017177e-05,
645
+ "loss": 6.3499,
646
+ "step": 45500
647
+ },
648
+ {
649
+ "epoch": 9.635525764558023,
650
+ "grad_norm": 3.5000336170196533,
651
+ "learning_rate": 4.036866359447005e-05,
652
+ "loss": 6.3199,
653
+ "step": 46000
654
+ },
655
+ {
656
+ "epoch": 9.74025974025974,
657
+ "grad_norm": 3.4947054386138916,
658
+ "learning_rate": 4.0263929618768334e-05,
659
+ "loss": 6.3474,
660
+ "step": 46500
661
+ },
662
+ {
663
+ "epoch": 9.844993715961458,
664
+ "grad_norm": 3.3658857345581055,
665
+ "learning_rate": 4.0159195643066614e-05,
666
+ "loss": 6.3296,
667
+ "step": 47000
668
+ },
669
+ {
670
+ "epoch": 9.949727691663176,
671
+ "grad_norm": 2.9811642169952393,
672
+ "learning_rate": 4.0054671135316294e-05,
673
+ "loss": 6.345,
674
+ "step": 47500
675
+ },
676
+ {
677
+ "epoch": 10.054461667364894,
678
+ "grad_norm": 4.165875434875488,
679
+ "learning_rate": 3.994993715961458e-05,
680
+ "loss": 6.3209,
681
+ "step": 48000
682
+ },
683
+ {
684
+ "epoch": 10.15919564306661,
685
+ "grad_norm": 3.6118202209472656,
686
+ "learning_rate": 3.9845203183912866e-05,
687
+ "loss": 6.3175,
688
+ "step": 48500
689
+ },
690
+ {
691
+ "epoch": 10.263929618768328,
692
+ "grad_norm": 3.930669069290161,
693
+ "learning_rate": 3.9740469208211146e-05,
694
+ "loss": 6.3231,
695
+ "step": 49000
696
+ },
697
+ {
698
+ "epoch": 10.368663594470046,
699
+ "grad_norm": 3.1688554286956787,
700
+ "learning_rate": 3.963594470046083e-05,
701
+ "loss": 6.309,
702
+ "step": 49500
703
+ },
704
+ {
705
+ "epoch": 10.473397570171764,
706
+ "grad_norm": 3.6746394634246826,
707
+ "learning_rate": 3.953121072475911e-05,
708
+ "loss": 6.3077,
709
+ "step": 50000
710
+ },
711
+ {
712
+ "epoch": 10.57813154587348,
713
+ "grad_norm": 3.5134785175323486,
714
+ "learning_rate": 3.942647674905739e-05,
715
+ "loss": 6.3299,
716
+ "step": 50500
717
+ },
718
+ {
719
+ "epoch": 10.682865521575199,
720
+ "grad_norm": 3.2903287410736084,
721
+ "learning_rate": 3.932174277335568e-05,
722
+ "loss": 6.3178,
723
+ "step": 51000
724
+ },
725
+ {
726
+ "epoch": 10.787599497276917,
727
+ "grad_norm": 3.5344769954681396,
728
+ "learning_rate": 3.921700879765396e-05,
729
+ "loss": 6.3139,
730
+ "step": 51500
731
+ },
732
+ {
733
+ "epoch": 10.892333472978635,
734
+ "grad_norm": 3.5710573196411133,
735
+ "learning_rate": 3.9112274821952245e-05,
736
+ "loss": 6.306,
737
+ "step": 52000
738
+ },
739
+ {
740
+ "epoch": 10.997067448680351,
741
+ "grad_norm": 3.4193336963653564,
742
+ "learning_rate": 3.9007540846250524e-05,
743
+ "loss": 6.3129,
744
+ "step": 52500
745
+ },
746
+ {
747
+ "epoch": 11.101801424382069,
748
+ "grad_norm": 3.683143377304077,
749
+ "learning_rate": 3.890301633850021e-05,
750
+ "loss": 6.3084,
751
+ "step": 53000
752
+ },
753
+ {
754
+ "epoch": 11.206535400083787,
755
+ "grad_norm": 3.214221239089966,
756
+ "learning_rate": 3.879828236279849e-05,
757
+ "loss": 6.302,
758
+ "step": 53500
759
+ },
760
+ {
761
+ "epoch": 11.311269375785505,
762
+ "grad_norm": 3.5691747665405273,
763
+ "learning_rate": 3.869354838709678e-05,
764
+ "loss": 6.3164,
765
+ "step": 54000
766
+ },
767
+ {
768
+ "epoch": 11.416003351487223,
769
+ "grad_norm": 3.2734036445617676,
770
+ "learning_rate": 3.858881441139506e-05,
771
+ "loss": 6.2889,
772
+ "step": 54500
773
+ },
774
+ {
775
+ "epoch": 11.52073732718894,
776
+ "grad_norm": 4.049854278564453,
777
+ "learning_rate": 3.848428990364474e-05,
778
+ "loss": 6.2957,
779
+ "step": 55000
780
+ },
781
+ {
782
+ "epoch": 11.625471302890658,
783
+ "grad_norm": 3.837921380996704,
784
+ "learning_rate": 3.837955592794303e-05,
785
+ "loss": 6.2806,
786
+ "step": 55500
787
+ },
788
+ {
789
+ "epoch": 11.730205278592376,
790
+ "grad_norm": 3.4606828689575195,
791
+ "learning_rate": 3.827482195224131e-05,
792
+ "loss": 6.2896,
793
+ "step": 56000
794
+ },
795
+ {
796
+ "epoch": 11.834939254294094,
797
+ "grad_norm": 4.859198093414307,
798
+ "learning_rate": 3.8170087976539596e-05,
799
+ "loss": 6.273,
800
+ "step": 56500
801
+ },
802
+ {
803
+ "epoch": 11.93967322999581,
804
+ "grad_norm": 4.689023494720459,
805
+ "learning_rate": 3.806535400083787e-05,
806
+ "loss": 6.2776,
807
+ "step": 57000
808
+ },
809
+ {
810
+ "epoch": 12.044407205697528,
811
+ "grad_norm": 4.234752178192139,
812
+ "learning_rate": 3.7960829493087555e-05,
813
+ "loss": 6.282,
814
+ "step": 57500
815
+ },
816
+ {
817
+ "epoch": 12.149141181399246,
818
+ "grad_norm": 3.950773239135742,
819
+ "learning_rate": 3.785609551738584e-05,
820
+ "loss": 6.283,
821
+ "step": 58000
822
+ },
823
+ {
824
+ "epoch": 12.253875157100964,
825
+ "grad_norm": 4.1780548095703125,
826
+ "learning_rate": 3.775136154168412e-05,
827
+ "loss": 6.2635,
828
+ "step": 58500
829
+ },
830
+ {
831
+ "epoch": 12.35860913280268,
832
+ "grad_norm": 3.2049672603607178,
833
+ "learning_rate": 3.764662756598241e-05,
834
+ "loss": 6.2858,
835
+ "step": 59000
836
+ },
837
+ {
838
+ "epoch": 12.463343108504398,
839
+ "grad_norm": 3.863649606704712,
840
+ "learning_rate": 3.7541893590280694e-05,
841
+ "loss": 6.2609,
842
+ "step": 59500
843
+ },
844
+ {
845
+ "epoch": 12.568077084206116,
846
+ "grad_norm": 3.881343364715576,
847
+ "learning_rate": 3.743715961457897e-05,
848
+ "loss": 6.2695,
849
+ "step": 60000
850
+ },
851
+ {
852
+ "epoch": 12.672811059907835,
853
+ "grad_norm": 3.522132635116577,
854
+ "learning_rate": 3.7332635106828654e-05,
855
+ "loss": 6.2523,
856
+ "step": 60500
857
+ },
858
+ {
859
+ "epoch": 12.777545035609553,
860
+ "grad_norm": 4.043595790863037,
861
+ "learning_rate": 3.722790113112694e-05,
862
+ "loss": 6.2546,
863
+ "step": 61000
864
+ },
865
+ {
866
+ "epoch": 12.882279011311269,
867
+ "grad_norm": 3.4860141277313232,
868
+ "learning_rate": 3.712316715542522e-05,
869
+ "loss": 6.2468,
870
+ "step": 61500
871
+ },
872
+ {
873
+ "epoch": 12.987012987012987,
874
+ "grad_norm": 3.9201574325561523,
875
+ "learning_rate": 3.7018433179723506e-05,
876
+ "loss": 6.2615,
877
+ "step": 62000
878
+ },
879
+ {
880
+ "epoch": 13.091746962714705,
881
+ "grad_norm": 3.5582118034362793,
882
+ "learning_rate": 3.6913699204021786e-05,
883
+ "loss": 6.2529,
884
+ "step": 62500
885
+ },
886
+ {
887
+ "epoch": 13.196480938416423,
888
+ "grad_norm": 3.1254093647003174,
889
+ "learning_rate": 3.680917469627147e-05,
890
+ "loss": 6.2418,
891
+ "step": 63000
892
+ },
893
+ {
894
+ "epoch": 13.30121491411814,
895
+ "grad_norm": 4.058616638183594,
896
+ "learning_rate": 3.670444072056975e-05,
897
+ "loss": 6.243,
898
+ "step": 63500
899
+ },
900
+ {
901
+ "epoch": 13.405948889819857,
902
+ "grad_norm": 3.5146963596343994,
903
+ "learning_rate": 3.659970674486803e-05,
904
+ "loss": 6.2595,
905
+ "step": 64000
906
+ },
907
+ {
908
+ "epoch": 13.510682865521575,
909
+ "grad_norm": 3.804818630218506,
910
+ "learning_rate": 3.649497276916632e-05,
911
+ "loss": 6.2438,
912
+ "step": 64500
913
+ },
914
+ {
915
+ "epoch": 13.615416841223293,
916
+ "grad_norm": 3.591214179992676,
917
+ "learning_rate": 3.6390238793464605e-05,
918
+ "loss": 6.2266,
919
+ "step": 65000
920
+ },
921
+ {
922
+ "epoch": 13.72015081692501,
923
+ "grad_norm": 4.973635196685791,
924
+ "learning_rate": 3.6285504817762885e-05,
925
+ "loss": 6.2501,
926
+ "step": 65500
927
+ },
928
+ {
929
+ "epoch": 13.824884792626728,
930
+ "grad_norm": 4.189575672149658,
931
+ "learning_rate": 3.618098031001257e-05,
932
+ "loss": 6.2341,
933
+ "step": 66000
934
+ },
935
+ {
936
+ "epoch": 13.929618768328446,
937
+ "grad_norm": 4.408186912536621,
938
+ "learning_rate": 3.607624633431085e-05,
939
+ "loss": 6.227,
940
+ "step": 66500
941
+ },
942
+ {
943
+ "epoch": 14.034352744030164,
944
+ "grad_norm": 4.066199779510498,
945
+ "learning_rate": 3.597151235860913e-05,
946
+ "loss": 6.2316,
947
+ "step": 67000
948
+ },
949
+ {
950
+ "epoch": 14.139086719731882,
951
+ "grad_norm": 3.8263683319091797,
952
+ "learning_rate": 3.586677838290742e-05,
953
+ "loss": 6.2322,
954
+ "step": 67500
955
+ },
956
+ {
957
+ "epoch": 14.243820695433598,
958
+ "grad_norm": 4.787020206451416,
959
+ "learning_rate": 3.57620444072057e-05,
960
+ "loss": 6.2099,
961
+ "step": 68000
962
+ },
963
+ {
964
+ "epoch": 14.348554671135316,
965
+ "grad_norm": 3.5196545124053955,
966
+ "learning_rate": 3.565751989945538e-05,
967
+ "loss": 6.2425,
968
+ "step": 68500
969
+ },
970
+ {
971
+ "epoch": 14.453288646837034,
972
+ "grad_norm": 4.1746439933776855,
973
+ "learning_rate": 3.555278592375367e-05,
974
+ "loss": 6.2373,
975
+ "step": 69000
976
+ },
977
+ {
978
+ "epoch": 14.558022622538752,
979
+ "grad_norm": 4.07820463180542,
980
+ "learning_rate": 3.544805194805195e-05,
981
+ "loss": 6.2099,
982
+ "step": 69500
983
+ },
984
+ {
985
+ "epoch": 14.662756598240469,
986
+ "grad_norm": 3.400038242340088,
987
+ "learning_rate": 3.534331797235023e-05,
988
+ "loss": 6.216,
989
+ "step": 70000
990
+ },
991
+ {
992
+ "epoch": 14.767490573942187,
993
+ "grad_norm": 4.578042030334473,
994
+ "learning_rate": 3.5238793464599916e-05,
995
+ "loss": 6.2091,
996
+ "step": 70500
997
+ },
998
+ {
999
+ "epoch": 14.872224549643905,
1000
+ "grad_norm": 3.6254208087921143,
1001
+ "learning_rate": 3.51340594888982e-05,
1002
+ "loss": 6.2258,
1003
+ "step": 71000
1004
+ },
1005
+ {
1006
+ "epoch": 14.976958525345623,
1007
+ "grad_norm": 3.496166467666626,
1008
+ "learning_rate": 3.502932551319648e-05,
1009
+ "loss": 6.2138,
1010
+ "step": 71500
1011
+ },
1012
+ {
1013
+ "epoch": 15.081692501047339,
1014
+ "grad_norm": 3.5367865562438965,
1015
+ "learning_rate": 3.492459153749477e-05,
1016
+ "loss": 6.213,
1017
+ "step": 72000
1018
+ },
1019
+ {
1020
+ "epoch": 15.186426476749057,
1021
+ "grad_norm": 3.4754440784454346,
1022
+ "learning_rate": 3.481985756179305e-05,
1023
+ "loss": 6.2153,
1024
+ "step": 72500
1025
+ },
1026
+ {
1027
+ "epoch": 15.291160452450775,
1028
+ "grad_norm": 4.432271957397461,
1029
+ "learning_rate": 3.4715333054042735e-05,
1030
+ "loss": 6.2007,
1031
+ "step": 73000
1032
+ },
1033
+ {
1034
+ "epoch": 15.395894428152493,
1035
+ "grad_norm": 3.8427770137786865,
1036
+ "learning_rate": 3.4610599078341014e-05,
1037
+ "loss": 6.2071,
1038
+ "step": 73500
1039
+ },
1040
+ {
1041
+ "epoch": 15.50062840385421,
1042
+ "grad_norm": 3.9617857933044434,
1043
+ "learning_rate": 3.4505865102639294e-05,
1044
+ "loss": 6.2142,
1045
+ "step": 74000
1046
+ },
1047
+ {
1048
+ "epoch": 15.605362379555928,
1049
+ "grad_norm": 3.769693613052368,
1050
+ "learning_rate": 3.440113112693758e-05,
1051
+ "loss": 6.2065,
1052
+ "step": 74500
1053
+ },
1054
+ {
1055
+ "epoch": 15.710096355257646,
1056
+ "grad_norm": 3.825507402420044,
1057
+ "learning_rate": 3.429639715123587e-05,
1058
+ "loss": 6.2072,
1059
+ "step": 75000
1060
+ },
1061
+ {
1062
+ "epoch": 15.814830330959364,
1063
+ "grad_norm": 3.982872724533081,
1064
+ "learning_rate": 3.4191872643485554e-05,
1065
+ "loss": 6.2003,
1066
+ "step": 75500
1067
+ },
1068
+ {
1069
+ "epoch": 15.91956430666108,
1070
+ "grad_norm": 3.9958648681640625,
1071
+ "learning_rate": 3.408713866778383e-05,
1072
+ "loss": 6.1913,
1073
+ "step": 76000
1074
+ },
1075
+ {
1076
+ "epoch": 16.0242982823628,
1077
+ "grad_norm": 3.947957754135132,
1078
+ "learning_rate": 3.398240469208211e-05,
1079
+ "loss": 6.2019,
1080
+ "step": 76500
1081
+ },
1082
+ {
1083
+ "epoch": 16.129032258064516,
1084
+ "grad_norm": 3.8135411739349365,
1085
+ "learning_rate": 3.387767071638039e-05,
1086
+ "loss": 6.1944,
1087
+ "step": 77000
1088
+ },
1089
+ {
1090
+ "epoch": 16.233766233766232,
1091
+ "grad_norm": 3.940861701965332,
1092
+ "learning_rate": 3.377293674067868e-05,
1093
+ "loss": 6.1893,
1094
+ "step": 77500
1095
+ },
1096
+ {
1097
+ "epoch": 16.338500209467952,
1098
+ "grad_norm": 5.24894905090332,
1099
+ "learning_rate": 3.3668412232928366e-05,
1100
+ "loss": 6.1984,
1101
+ "step": 78000
1102
+ },
1103
+ {
1104
+ "epoch": 16.44323418516967,
1105
+ "grad_norm": 4.470870494842529,
1106
+ "learning_rate": 3.3563678257226645e-05,
1107
+ "loss": 6.1958,
1108
+ "step": 78500
1109
+ },
1110
+ {
1111
+ "epoch": 16.547968160871388,
1112
+ "grad_norm": 3.699892282485962,
1113
+ "learning_rate": 3.345894428152493e-05,
1114
+ "loss": 6.1952,
1115
+ "step": 79000
1116
+ },
1117
+ {
1118
+ "epoch": 16.652702136573104,
1119
+ "grad_norm": 4.136711120605469,
1120
+ "learning_rate": 3.335421030582321e-05,
1121
+ "loss": 6.1896,
1122
+ "step": 79500
1123
+ },
1124
+ {
1125
+ "epoch": 16.75743611227482,
1126
+ "grad_norm": 4.904257297515869,
1127
+ "learning_rate": 3.324968579807289e-05,
1128
+ "loss": 6.1715,
1129
+ "step": 80000
1130
+ },
1131
+ {
1132
+ "epoch": 16.86217008797654,
1133
+ "grad_norm": 4.219280242919922,
1134
+ "learning_rate": 3.314495182237118e-05,
1135
+ "loss": 6.1829,
1136
+ "step": 80500
1137
+ },
1138
+ {
1139
+ "epoch": 16.966904063678257,
1140
+ "grad_norm": 4.426414489746094,
1141
+ "learning_rate": 3.3040217846669464e-05,
1142
+ "loss": 6.1782,
1143
+ "step": 81000
1144
+ },
1145
+ {
1146
+ "epoch": 17.071638039379973,
1147
+ "grad_norm": 4.792020797729492,
1148
+ "learning_rate": 3.2935483870967744e-05,
1149
+ "loss": 6.1675,
1150
+ "step": 81500
1151
+ },
1152
+ {
1153
+ "epoch": 17.176372015081693,
1154
+ "grad_norm": 3.9796903133392334,
1155
+ "learning_rate": 3.283095936321743e-05,
1156
+ "loss": 6.179,
1157
+ "step": 82000
1158
+ },
1159
+ {
1160
+ "epoch": 17.28110599078341,
1161
+ "grad_norm": 4.554388046264648,
1162
+ "learning_rate": 3.272622538751571e-05,
1163
+ "loss": 6.1756,
1164
+ "step": 82500
1165
+ },
1166
+ {
1167
+ "epoch": 17.38583996648513,
1168
+ "grad_norm": 4.024316787719727,
1169
+ "learning_rate": 3.262149141181399e-05,
1170
+ "loss": 6.177,
1171
+ "step": 83000
1172
+ },
1173
+ {
1174
+ "epoch": 17.490573942186845,
1175
+ "grad_norm": 4.059772968292236,
1176
+ "learning_rate": 3.2516757436112276e-05,
1177
+ "loss": 6.1818,
1178
+ "step": 83500
1179
+ },
1180
+ {
1181
+ "epoch": 17.59530791788856,
1182
+ "grad_norm": 4.296391487121582,
1183
+ "learning_rate": 3.241223292836196e-05,
1184
+ "loss": 6.1866,
1185
+ "step": 84000
1186
+ },
1187
+ {
1188
+ "epoch": 17.70004189359028,
1189
+ "grad_norm": 4.008220672607422,
1190
+ "learning_rate": 3.230749895266024e-05,
1191
+ "loss": 6.172,
1192
+ "step": 84500
1193
+ },
1194
+ {
1195
+ "epoch": 17.804775869291998,
1196
+ "grad_norm": 4.639082908630371,
1197
+ "learning_rate": 3.220276497695853e-05,
1198
+ "loss": 6.169,
1199
+ "step": 85000
1200
+ },
1201
+ {
1202
+ "epoch": 17.909509844993718,
1203
+ "grad_norm": 4.635848522186279,
1204
+ "learning_rate": 3.209803100125681e-05,
1205
+ "loss": 6.1721,
1206
+ "step": 85500
1207
+ },
1208
+ {
1209
+ "epoch": 18.014243820695434,
1210
+ "grad_norm": 4.662270545959473,
1211
+ "learning_rate": 3.199329702555509e-05,
1212
+ "loss": 6.1575,
1213
+ "step": 86000
1214
+ },
1215
+ {
1216
+ "epoch": 18.11897779639715,
1217
+ "grad_norm": 4.280701637268066,
1218
+ "learning_rate": 3.1888772517804775e-05,
1219
+ "loss": 6.1482,
1220
+ "step": 86500
1221
+ },
1222
+ {
1223
+ "epoch": 18.22371177209887,
1224
+ "grad_norm": 3.8602380752563477,
1225
+ "learning_rate": 3.178403854210306e-05,
1226
+ "loss": 6.1565,
1227
+ "step": 87000
1228
+ },
1229
+ {
1230
+ "epoch": 18.328445747800586,
1231
+ "grad_norm": 4.634263515472412,
1232
+ "learning_rate": 3.167930456640134e-05,
1233
+ "loss": 6.1587,
1234
+ "step": 87500
1235
+ },
1236
+ {
1237
+ "epoch": 18.433179723502302,
1238
+ "grad_norm": 4.115392208099365,
1239
+ "learning_rate": 3.157457059069963e-05,
1240
+ "loss": 6.1436,
1241
+ "step": 88000
1242
+ },
1243
+ {
1244
+ "epoch": 18.537913699204022,
1245
+ "grad_norm": 3.6665916442871094,
1246
+ "learning_rate": 3.146983661499791e-05,
1247
+ "loss": 6.1442,
1248
+ "step": 88500
1249
+ },
1250
+ {
1251
+ "epoch": 18.64264767490574,
1252
+ "grad_norm": 4.444345951080322,
1253
+ "learning_rate": 3.1365312107247594e-05,
1254
+ "loss": 6.1571,
1255
+ "step": 89000
1256
+ },
1257
+ {
1258
+ "epoch": 18.74738165060746,
1259
+ "grad_norm": 3.792792558670044,
1260
+ "learning_rate": 3.1260578131545873e-05,
1261
+ "loss": 6.1535,
1262
+ "step": 89500
1263
+ },
1264
+ {
1265
+ "epoch": 18.852115626309175,
1266
+ "grad_norm": 3.904019832611084,
1267
+ "learning_rate": 3.115584415584415e-05,
1268
+ "loss": 6.1501,
1269
+ "step": 90000
1270
+ },
1271
+ {
1272
+ "epoch": 18.95684960201089,
1273
+ "grad_norm": 4.531284332275391,
1274
+ "learning_rate": 3.105111018014244e-05,
1275
+ "loss": 6.1592,
1276
+ "step": 90500
1277
+ },
1278
+ {
1279
+ "epoch": 19.06158357771261,
1280
+ "grad_norm": 3.7976317405700684,
1281
+ "learning_rate": 3.0946376204440726e-05,
1282
+ "loss": 6.1474,
1283
+ "step": 91000
1284
+ },
1285
+ {
1286
+ "epoch": 19.166317553414327,
1287
+ "grad_norm": 3.8021469116210938,
1288
+ "learning_rate": 3.084185169669041e-05,
1289
+ "loss": 6.1408,
1290
+ "step": 91500
1291
+ },
1292
+ {
1293
+ "epoch": 19.271051529116047,
1294
+ "grad_norm": 4.194758892059326,
1295
+ "learning_rate": 3.073711772098869e-05,
1296
+ "loss": 6.1476,
1297
+ "step": 92000
1298
+ },
1299
+ {
1300
+ "epoch": 19.375785504817763,
1301
+ "grad_norm": 4.084668159484863,
1302
+ "learning_rate": 3.063238374528697e-05,
1303
+ "loss": 6.1443,
1304
+ "step": 92500
1305
+ },
1306
+ {
1307
+ "epoch": 19.48051948051948,
1308
+ "grad_norm": 4.383222579956055,
1309
+ "learning_rate": 3.052764976958525e-05,
1310
+ "loss": 6.1422,
1311
+ "step": 93000
1312
+ },
1313
+ {
1314
+ "epoch": 19.5852534562212,
1315
+ "grad_norm": 4.250995635986328,
1316
+ "learning_rate": 3.042312526183494e-05,
1317
+ "loss": 6.1375,
1318
+ "step": 93500
1319
+ },
1320
+ {
1321
+ "epoch": 19.689987431922916,
1322
+ "grad_norm": 4.78529691696167,
1323
+ "learning_rate": 3.0318391286133225e-05,
1324
+ "loss": 6.1368,
1325
+ "step": 94000
1326
+ },
1327
+ {
1328
+ "epoch": 19.794721407624632,
1329
+ "grad_norm": 3.4997754096984863,
1330
+ "learning_rate": 3.0213657310431504e-05,
1331
+ "loss": 6.1432,
1332
+ "step": 94500
1333
+ },
1334
+ {
1335
+ "epoch": 19.89945538332635,
1336
+ "grad_norm": 4.723648548126221,
1337
+ "learning_rate": 3.0108923334729787e-05,
1338
+ "loss": 6.1263,
1339
+ "step": 95000
1340
+ },
1341
+ {
1342
+ "epoch": 20.004189359028068,
1343
+ "grad_norm": 3.930859088897705,
1344
+ "learning_rate": 3.0004398826979474e-05,
1345
+ "loss": 6.1387,
1346
+ "step": 95500
1347
+ },
1348
+ {
1349
+ "epoch": 20.108923334729788,
1350
+ "grad_norm": 4.286599159240723,
1351
+ "learning_rate": 2.9899664851277754e-05,
1352
+ "loss": 6.1187,
1353
+ "step": 96000
1354
+ },
1355
+ {
1356
+ "epoch": 20.213657310431504,
1357
+ "grad_norm": 3.8475680351257324,
1358
+ "learning_rate": 2.9794930875576037e-05,
1359
+ "loss": 6.1312,
1360
+ "step": 96500
1361
+ },
1362
+ {
1363
+ "epoch": 20.31839128613322,
1364
+ "grad_norm": 4.844906806945801,
1365
+ "learning_rate": 2.9690196899874323e-05,
1366
+ "loss": 6.1457,
1367
+ "step": 97000
1368
+ },
1369
+ {
1370
+ "epoch": 20.42312526183494,
1371
+ "grad_norm": 4.691315174102783,
1372
+ "learning_rate": 2.958567239212401e-05,
1373
+ "loss": 6.1351,
1374
+ "step": 97500
1375
+ },
1376
+ {
1377
+ "epoch": 20.527859237536656,
1378
+ "grad_norm": 6.15250825881958,
1379
+ "learning_rate": 2.9480938416422286e-05,
1380
+ "loss": 6.1275,
1381
+ "step": 98000
1382
+ },
1383
+ {
1384
+ "epoch": 20.632593213238373,
1385
+ "grad_norm": 3.8872599601745605,
1386
+ "learning_rate": 2.9376204440720573e-05,
1387
+ "loss": 6.1275,
1388
+ "step": 98500
1389
+ },
1390
+ {
1391
+ "epoch": 20.737327188940093,
1392
+ "grad_norm": 4.541051864624023,
1393
+ "learning_rate": 2.9271470465018852e-05,
1394
+ "loss": 6.1472,
1395
+ "step": 99000
1396
+ },
1397
+ {
1398
+ "epoch": 20.84206116464181,
1399
+ "grad_norm": 4.556408405303955,
1400
+ "learning_rate": 2.9166736489317135e-05,
1401
+ "loss": 6.1369,
1402
+ "step": 99500
1403
+ },
1404
+ {
1405
+ "epoch": 20.94679514034353,
1406
+ "grad_norm": 4.5567498207092285,
1407
+ "learning_rate": 2.9062211981566822e-05,
1408
+ "loss": 6.1148,
1409
+ "step": 100000
1410
+ },
1411
+ {
1412
+ "epoch": 21.051529116045245,
1413
+ "grad_norm": 4.647518634796143,
1414
+ "learning_rate": 2.8957478005865102e-05,
1415
+ "loss": 6.1281,
1416
+ "step": 100500
1417
+ },
1418
+ {
1419
+ "epoch": 21.15626309174696,
1420
+ "grad_norm": 4.372421741485596,
1421
+ "learning_rate": 2.8852744030163388e-05,
1422
+ "loss": 6.1226,
1423
+ "step": 101000
1424
+ },
1425
+ {
1426
+ "epoch": 21.26099706744868,
1427
+ "grad_norm": 4.270533084869385,
1428
+ "learning_rate": 2.874801005446167e-05,
1429
+ "loss": 6.123,
1430
+ "step": 101500
1431
+ },
1432
+ {
1433
+ "epoch": 21.365731043150397,
1434
+ "grad_norm": 3.5596370697021484,
1435
+ "learning_rate": 2.8643485546711358e-05,
1436
+ "loss": 6.1402,
1437
+ "step": 102000
1438
+ },
1439
+ {
1440
+ "epoch": 21.470465018852117,
1441
+ "grad_norm": 5.230384826660156,
1442
+ "learning_rate": 2.8538751571009638e-05,
1443
+ "loss": 6.1199,
1444
+ "step": 102500
1445
+ },
1446
+ {
1447
+ "epoch": 21.575198994553833,
1448
+ "grad_norm": 3.9881417751312256,
1449
+ "learning_rate": 2.843401759530792e-05,
1450
+ "loss": 6.1244,
1451
+ "step": 103000
1452
+ },
1453
+ {
1454
+ "epoch": 21.67993297025555,
1455
+ "grad_norm": 4.617568016052246,
1456
+ "learning_rate": 2.83292836196062e-05,
1457
+ "loss": 6.1154,
1458
+ "step": 103500
1459
+ },
1460
+ {
1461
+ "epoch": 21.78466694595727,
1462
+ "grad_norm": 4.641009330749512,
1463
+ "learning_rate": 2.8224759111855887e-05,
1464
+ "loss": 6.113,
1465
+ "step": 104000
1466
+ },
1467
+ {
1468
+ "epoch": 21.889400921658986,
1469
+ "grad_norm": 4.005772113800049,
1470
+ "learning_rate": 2.812002513615417e-05,
1471
+ "loss": 6.1163,
1472
+ "step": 104500
1473
+ },
1474
+ {
1475
+ "epoch": 21.994134897360702,
1476
+ "grad_norm": 4.2611799240112305,
1477
+ "learning_rate": 2.801529116045245e-05,
1478
+ "loss": 6.1023,
1479
+ "step": 105000
1480
+ },
1481
+ {
1482
+ "epoch": 22.098868873062422,
1483
+ "grad_norm": 4.568357467651367,
1484
+ "learning_rate": 2.7910557184750736e-05,
1485
+ "loss": 6.1169,
1486
+ "step": 105500
1487
+ },
1488
+ {
1489
+ "epoch": 22.203602848764138,
1490
+ "grad_norm": 4.323103427886963,
1491
+ "learning_rate": 2.780603267700042e-05,
1492
+ "loss": 6.1226,
1493
+ "step": 106000
1494
+ },
1495
+ {
1496
+ "epoch": 22.308336824465858,
1497
+ "grad_norm": 4.507444381713867,
1498
+ "learning_rate": 2.77012987012987e-05,
1499
+ "loss": 6.1052,
1500
+ "step": 106500
1501
+ },
1502
+ {
1503
+ "epoch": 22.413070800167574,
1504
+ "grad_norm": 4.301244735717773,
1505
+ "learning_rate": 2.7596564725596985e-05,
1506
+ "loss": 6.0944,
1507
+ "step": 107000
1508
+ },
1509
+ {
1510
+ "epoch": 22.51780477586929,
1511
+ "grad_norm": 4.984853267669678,
1512
+ "learning_rate": 2.749183074989527e-05,
1513
+ "loss": 6.1125,
1514
+ "step": 107500
1515
+ },
1516
+ {
1517
+ "epoch": 22.62253875157101,
1518
+ "grad_norm": 4.682931423187256,
1519
+ "learning_rate": 2.7387096774193548e-05,
1520
+ "loss": 6.1158,
1521
+ "step": 108000
1522
+ },
1523
+ {
1524
+ "epoch": 22.727272727272727,
1525
+ "grad_norm": 4.494015693664551,
1526
+ "learning_rate": 2.7282572266443235e-05,
1527
+ "loss": 6.1035,
1528
+ "step": 108500
1529
+ },
1530
+ {
1531
+ "epoch": 22.832006702974446,
1532
+ "grad_norm": 3.880779981613159,
1533
+ "learning_rate": 2.717783829074152e-05,
1534
+ "loss": 6.1084,
1535
+ "step": 109000
1536
+ },
1537
+ {
1538
+ "epoch": 22.936740678676163,
1539
+ "grad_norm": 4.154653072357178,
1540
+ "learning_rate": 2.7073104315039798e-05,
1541
+ "loss": 6.0945,
1542
+ "step": 109500
1543
+ },
1544
+ {
1545
+ "epoch": 23.04147465437788,
1546
+ "grad_norm": 5.443271160125732,
1547
+ "learning_rate": 2.6968370339338084e-05,
1548
+ "loss": 6.1016,
1549
+ "step": 110000
1550
+ },
1551
+ {
1552
+ "epoch": 23.1462086300796,
1553
+ "grad_norm": 4.298133373260498,
1554
+ "learning_rate": 2.686384583158777e-05,
1555
+ "loss": 6.103,
1556
+ "step": 110500
1557
+ },
1558
+ {
1559
+ "epoch": 23.250942605781315,
1560
+ "grad_norm": 4.379884243011475,
1561
+ "learning_rate": 2.675911185588605e-05,
1562
+ "loss": 6.0814,
1563
+ "step": 111000
1564
+ },
1565
+ {
1566
+ "epoch": 23.35567658148303,
1567
+ "grad_norm": 6.175398349761963,
1568
+ "learning_rate": 2.6654377880184333e-05,
1569
+ "loss": 6.1088,
1570
+ "step": 111500
1571
+ },
1572
+ {
1573
+ "epoch": 23.46041055718475,
1574
+ "grad_norm": 4.121715068817139,
1575
+ "learning_rate": 2.6549643904482613e-05,
1576
+ "loss": 6.0966,
1577
+ "step": 112000
1578
+ },
1579
+ {
1580
+ "epoch": 23.565144532886467,
1581
+ "grad_norm": 5.040287494659424,
1582
+ "learning_rate": 2.6444909928780896e-05,
1583
+ "loss": 6.098,
1584
+ "step": 112500
1585
+ },
1586
+ {
1587
+ "epoch": 23.669878508588187,
1588
+ "grad_norm": 4.766879081726074,
1589
+ "learning_rate": 2.6340175953079182e-05,
1590
+ "loss": 6.0957,
1591
+ "step": 113000
1592
+ },
1593
+ {
1594
+ "epoch": 23.774612484289904,
1595
+ "grad_norm": 5.87930965423584,
1596
+ "learning_rate": 2.623565144532887e-05,
1597
+ "loss": 6.1089,
1598
+ "step": 113500
1599
+ },
1600
+ {
1601
+ "epoch": 23.87934645999162,
1602
+ "grad_norm": 5.318653583526611,
1603
+ "learning_rate": 2.613091746962715e-05,
1604
+ "loss": 6.0761,
1605
+ "step": 114000
1606
+ },
1607
+ {
1608
+ "epoch": 23.98408043569334,
1609
+ "grad_norm": 4.465319633483887,
1610
+ "learning_rate": 2.6026183493925432e-05,
1611
+ "loss": 6.0826,
1612
+ "step": 114500
1613
+ },
1614
+ {
1615
+ "epoch": 24.088814411395056,
1616
+ "grad_norm": 4.640571594238281,
1617
+ "learning_rate": 2.592144951822371e-05,
1618
+ "loss": 6.0805,
1619
+ "step": 115000
1620
+ },
1621
+ {
1622
+ "epoch": 24.193548387096776,
1623
+ "grad_norm": 4.252554416656494,
1624
+ "learning_rate": 2.5816925010473398e-05,
1625
+ "loss": 6.0701,
1626
+ "step": 115500
1627
+ },
1628
+ {
1629
+ "epoch": 24.298282362798492,
1630
+ "grad_norm": 4.704644203186035,
1631
+ "learning_rate": 2.571219103477168e-05,
1632
+ "loss": 6.0936,
1633
+ "step": 116000
1634
+ },
1635
+ {
1636
+ "epoch": 24.40301633850021,
1637
+ "grad_norm": 4.601324558258057,
1638
+ "learning_rate": 2.560745705906996e-05,
1639
+ "loss": 6.0758,
1640
+ "step": 116500
1641
+ },
1642
+ {
1643
+ "epoch": 24.507750314201928,
1644
+ "grad_norm": 4.380444526672363,
1645
+ "learning_rate": 2.5502723083368247e-05,
1646
+ "loss": 6.0871,
1647
+ "step": 117000
1648
+ },
1649
+ {
1650
+ "epoch": 24.612484289903644,
1651
+ "grad_norm": 4.119806289672852,
1652
+ "learning_rate": 2.5397989107666527e-05,
1653
+ "loss": 6.102,
1654
+ "step": 117500
1655
+ },
1656
+ {
1657
+ "epoch": 24.71721826560536,
1658
+ "grad_norm": 3.9712698459625244,
1659
+ "learning_rate": 2.5293464599916217e-05,
1660
+ "loss": 6.0999,
1661
+ "step": 118000
1662
+ },
1663
+ {
1664
+ "epoch": 24.82195224130708,
1665
+ "grad_norm": 5.146612167358398,
1666
+ "learning_rate": 2.5188730624214497e-05,
1667
+ "loss": 6.0947,
1668
+ "step": 118500
1669
+ },
1670
+ {
1671
+ "epoch": 24.926686217008797,
1672
+ "grad_norm": 4.406741142272949,
1673
+ "learning_rate": 2.508399664851278e-05,
1674
+ "loss": 6.0829,
1675
+ "step": 119000
1676
+ },
1677
+ {
1678
+ "epoch": 25.031420192710517,
1679
+ "grad_norm": 5.4739766120910645,
1680
+ "learning_rate": 2.497926267281106e-05,
1681
+ "loss": 6.0598,
1682
+ "step": 119500
1683
+ },
1684
+ {
1685
+ "epoch": 25.136154168412233,
1686
+ "grad_norm": 4.6231794357299805,
1687
+ "learning_rate": 2.4874738165060746e-05,
1688
+ "loss": 6.072,
1689
+ "step": 120000
1690
+ },
1691
+ {
1692
+ "epoch": 25.24088814411395,
1693
+ "grad_norm": 4.47750186920166,
1694
+ "learning_rate": 2.477000418935903e-05,
1695
+ "loss": 6.0664,
1696
+ "step": 120500
1697
+ },
1698
+ {
1699
+ "epoch": 25.34562211981567,
1700
+ "grad_norm": 5.023014068603516,
1701
+ "learning_rate": 2.4665270213657312e-05,
1702
+ "loss": 6.0894,
1703
+ "step": 121000
1704
+ },
1705
+ {
1706
+ "epoch": 25.450356095517385,
1707
+ "grad_norm": 5.687644004821777,
1708
+ "learning_rate": 2.4560536237955595e-05,
1709
+ "loss": 6.0936,
1710
+ "step": 121500
1711
+ },
1712
+ {
1713
+ "epoch": 25.555090071219105,
1714
+ "grad_norm": 4.534958362579346,
1715
+ "learning_rate": 2.4455802262253878e-05,
1716
+ "loss": 6.0794,
1717
+ "step": 122000
1718
+ },
1719
+ {
1720
+ "epoch": 25.65982404692082,
1721
+ "grad_norm": 5.563751697540283,
1722
+ "learning_rate": 2.435127775450356e-05,
1723
+ "loss": 6.081,
1724
+ "step": 122500
1725
+ },
1726
+ {
1727
+ "epoch": 25.764558022622538,
1728
+ "grad_norm": 4.613626956939697,
1729
+ "learning_rate": 2.4246543778801845e-05,
1730
+ "loss": 6.0982,
1731
+ "step": 123000
1732
+ },
1733
+ {
1734
+ "epoch": 25.869291998324258,
1735
+ "grad_norm": 4.645818710327148,
1736
+ "learning_rate": 2.4141809803100128e-05,
1737
+ "loss": 6.0665,
1738
+ "step": 123500
1739
+ },
1740
+ {
1741
+ "epoch": 25.974025974025974,
1742
+ "grad_norm": 4.9156928062438965,
1743
+ "learning_rate": 2.4037075827398407e-05,
1744
+ "loss": 6.0674,
1745
+ "step": 124000
1746
+ },
1747
+ {
1748
+ "epoch": 26.07875994972769,
1749
+ "grad_norm": 4.7342305183410645,
1750
+ "learning_rate": 2.3932551319648094e-05,
1751
+ "loss": 6.0741,
1752
+ "step": 124500
1753
+ },
1754
+ {
1755
+ "epoch": 26.18349392542941,
1756
+ "grad_norm": 4.607081413269043,
1757
+ "learning_rate": 2.3827817343946377e-05,
1758
+ "loss": 6.0626,
1759
+ "step": 125000
1760
+ },
1761
+ {
1762
+ "epoch": 26.288227901131126,
1763
+ "grad_norm": 4.820442199707031,
1764
+ "learning_rate": 2.372308336824466e-05,
1765
+ "loss": 6.0936,
1766
+ "step": 125500
1767
+ },
1768
+ {
1769
+ "epoch": 26.392961876832846,
1770
+ "grad_norm": 4.549975395202637,
1771
+ "learning_rate": 2.3618349392542943e-05,
1772
+ "loss": 6.0804,
1773
+ "step": 126000
1774
+ },
1775
+ {
1776
+ "epoch": 26.497695852534562,
1777
+ "grad_norm": 4.722150802612305,
1778
+ "learning_rate": 2.351382488479263e-05,
1779
+ "loss": 6.0555,
1780
+ "step": 126500
1781
+ },
1782
+ {
1783
+ "epoch": 26.60242982823628,
1784
+ "grad_norm": 4.2948408126831055,
1785
+ "learning_rate": 2.340909090909091e-05,
1786
+ "loss": 6.0626,
1787
+ "step": 127000
1788
+ },
1789
+ {
1790
+ "epoch": 26.707163803938,
1791
+ "grad_norm": 4.246878623962402,
1792
+ "learning_rate": 2.3304356933389193e-05,
1793
+ "loss": 6.0577,
1794
+ "step": 127500
1795
+ },
1796
+ {
1797
+ "epoch": 26.811897779639715,
1798
+ "grad_norm": 4.165809154510498,
1799
+ "learning_rate": 2.3199622957687476e-05,
1800
+ "loss": 6.0608,
1801
+ "step": 128000
1802
+ },
1803
+ {
1804
+ "epoch": 26.916631755341434,
1805
+ "grad_norm": 4.3159894943237305,
1806
+ "learning_rate": 2.309488898198576e-05,
1807
+ "loss": 6.0806,
1808
+ "step": 128500
1809
+ },
1810
+ {
1811
+ "epoch": 27.02136573104315,
1812
+ "grad_norm": 4.15300989151001,
1813
+ "learning_rate": 2.2990364474235442e-05,
1814
+ "loss": 6.0654,
1815
+ "step": 129000
1816
+ },
1817
+ {
1818
+ "epoch": 27.126099706744867,
1819
+ "grad_norm": 4.730154991149902,
1820
+ "learning_rate": 2.2885630498533725e-05,
1821
+ "loss": 6.0567,
1822
+ "step": 129500
1823
+ },
1824
+ {
1825
+ "epoch": 27.230833682446587,
1826
+ "grad_norm": 4.300974369049072,
1827
+ "learning_rate": 2.2780896522832008e-05,
1828
+ "loss": 6.0449,
1829
+ "step": 130000
1830
+ },
1831
+ {
1832
+ "epoch": 27.335567658148303,
1833
+ "grad_norm": 4.148283958435059,
1834
+ "learning_rate": 2.2676162547130288e-05,
1835
+ "loss": 6.0491,
1836
+ "step": 130500
1837
+ },
1838
+ {
1839
+ "epoch": 27.44030163385002,
1840
+ "grad_norm": 4.9924421310424805,
1841
+ "learning_rate": 2.2571638039379974e-05,
1842
+ "loss": 6.0491,
1843
+ "step": 131000
1844
+ },
1845
+ {
1846
+ "epoch": 27.54503560955174,
1847
+ "grad_norm": 5.713706016540527,
1848
+ "learning_rate": 2.246690406367826e-05,
1849
+ "loss": 6.0396,
1850
+ "step": 131500
1851
+ },
1852
+ {
1853
+ "epoch": 27.649769585253456,
1854
+ "grad_norm": 5.007369518280029,
1855
+ "learning_rate": 2.236217008797654e-05,
1856
+ "loss": 6.0378,
1857
+ "step": 132000
1858
+ },
1859
+ {
1860
+ "epoch": 27.754503560955175,
1861
+ "grad_norm": 4.500640392303467,
1862
+ "learning_rate": 2.2257436112274823e-05,
1863
+ "loss": 6.0313,
1864
+ "step": 132500
1865
+ },
1866
+ {
1867
+ "epoch": 27.85923753665689,
1868
+ "grad_norm": 4.709275722503662,
1869
+ "learning_rate": 2.2152702136573107e-05,
1870
+ "loss": 6.0278,
1871
+ "step": 133000
1872
+ },
1873
+ {
1874
+ "epoch": 27.963971512358608,
1875
+ "grad_norm": 4.891386032104492,
1876
+ "learning_rate": 2.204817762882279e-05,
1877
+ "loss": 6.0267,
1878
+ "step": 133500
1879
+ },
1880
+ {
1881
+ "epoch": 28.068705488060328,
1882
+ "grad_norm": 4.82666540145874,
1883
+ "learning_rate": 2.1943443653121073e-05,
1884
+ "loss": 5.986,
1885
+ "step": 134000
1886
+ },
1887
+ {
1888
+ "epoch": 28.173439463762044,
1889
+ "grad_norm": 4.489607810974121,
1890
+ "learning_rate": 2.1838709677419356e-05,
1891
+ "loss": 6.0209,
1892
+ "step": 134500
1893
+ },
1894
+ {
1895
+ "epoch": 28.278173439463764,
1896
+ "grad_norm": 4.719301700592041,
1897
+ "learning_rate": 2.173397570171764e-05,
1898
+ "loss": 5.9998,
1899
+ "step": 135000
1900
+ },
1901
+ {
1902
+ "epoch": 28.38290741516548,
1903
+ "grad_norm": 5.639565467834473,
1904
+ "learning_rate": 2.1629451193967322e-05,
1905
+ "loss": 5.9881,
1906
+ "step": 135500
1907
+ },
1908
+ {
1909
+ "epoch": 28.487641390867196,
1910
+ "grad_norm": 4.745512008666992,
1911
+ "learning_rate": 2.1524717218265605e-05,
1912
+ "loss": 6.0002,
1913
+ "step": 136000
1914
+ },
1915
+ {
1916
+ "epoch": 28.592375366568916,
1917
+ "grad_norm": 5.661725997924805,
1918
+ "learning_rate": 2.141998324256389e-05,
1919
+ "loss": 5.9967,
1920
+ "step": 136500
1921
+ },
1922
+ {
1923
+ "epoch": 28.697109342270632,
1924
+ "grad_norm": 5.3391194343566895,
1925
+ "learning_rate": 2.131524926686217e-05,
1926
+ "loss": 6.0024,
1927
+ "step": 137000
1928
+ },
1929
+ {
1930
+ "epoch": 28.80184331797235,
1931
+ "grad_norm": 5.1614089012146,
1932
+ "learning_rate": 2.1210724759111858e-05,
1933
+ "loss": 5.9992,
1934
+ "step": 137500
1935
+ },
1936
+ {
1937
+ "epoch": 28.90657729367407,
1938
+ "grad_norm": 5.429248332977295,
1939
+ "learning_rate": 2.110599078341014e-05,
1940
+ "loss": 5.9929,
1941
+ "step": 138000
1942
+ },
1943
+ {
1944
+ "epoch": 29.011311269375785,
1945
+ "grad_norm": 5.1270012855529785,
1946
+ "learning_rate": 2.100125680770842e-05,
1947
+ "loss": 5.9883,
1948
+ "step": 138500
1949
+ },
1950
+ {
1951
+ "epoch": 29.116045245077505,
1952
+ "grad_norm": 5.027891159057617,
1953
+ "learning_rate": 2.0896522832006704e-05,
1954
+ "loss": 5.963,
1955
+ "step": 139000
1956
+ },
1957
+ {
1958
+ "epoch": 29.22077922077922,
1959
+ "grad_norm": 5.712099552154541,
1960
+ "learning_rate": 2.079199832425639e-05,
1961
+ "loss": 5.9811,
1962
+ "step": 139500
1963
+ },
1964
+ {
1965
+ "epoch": 29.325513196480937,
1966
+ "grad_norm": 4.954220294952393,
1967
+ "learning_rate": 2.0687264348554674e-05,
1968
+ "loss": 5.9808,
1969
+ "step": 140000
1970
+ },
1971
+ {
1972
+ "epoch": 29.430247172182657,
1973
+ "grad_norm": 5.713419437408447,
1974
+ "learning_rate": 2.0582530372852953e-05,
1975
+ "loss": 5.9887,
1976
+ "step": 140500
1977
+ },
1978
+ {
1979
+ "epoch": 29.534981147884373,
1980
+ "grad_norm": 4.683711528778076,
1981
+ "learning_rate": 2.0477796397151236e-05,
1982
+ "loss": 5.9612,
1983
+ "step": 141000
1984
+ },
1985
+ {
1986
+ "epoch": 29.63971512358609,
1987
+ "grad_norm": 5.164538383483887,
1988
+ "learning_rate": 2.0373271889400923e-05,
1989
+ "loss": 5.993,
1990
+ "step": 141500
1991
+ },
1992
+ {
1993
+ "epoch": 29.74444909928781,
1994
+ "grad_norm": 5.386078357696533,
1995
+ "learning_rate": 2.0268537913699203e-05,
1996
+ "loss": 5.9735,
1997
+ "step": 142000
1998
+ },
1999
+ {
2000
+ "epoch": 29.849183074989526,
2001
+ "grad_norm": 4.4406418800354,
2002
+ "learning_rate": 2.016380393799749e-05,
2003
+ "loss": 5.9672,
2004
+ "step": 142500
2005
+ },
2006
+ {
2007
+ "epoch": 29.953917050691246,
2008
+ "grad_norm": 5.029815673828125,
2009
+ "learning_rate": 2.0059069962295772e-05,
2010
+ "loss": 5.961,
2011
+ "step": 143000
2012
+ },
2013
+ {
2014
+ "epoch": 30.058651026392962,
2015
+ "grad_norm": 4.666591167449951,
2016
+ "learning_rate": 1.9954335986594052e-05,
2017
+ "loss": 5.9505,
2018
+ "step": 143500
2019
+ },
2020
+ {
2021
+ "epoch": 30.163385002094678,
2022
+ "grad_norm": 6.975547790527344,
2023
+ "learning_rate": 1.9849602010892335e-05,
2024
+ "loss": 5.956,
2025
+ "step": 144000
2026
+ },
2027
+ {
2028
+ "epoch": 30.268118977796398,
2029
+ "grad_norm": 4.687684535980225,
2030
+ "learning_rate": 1.974507750314202e-05,
2031
+ "loss": 5.9475,
2032
+ "step": 144500
2033
+ },
2034
+ {
2035
+ "epoch": 30.372852953498114,
2036
+ "grad_norm": 5.594231605529785,
2037
+ "learning_rate": 1.96403435274403e-05,
2038
+ "loss": 5.9496,
2039
+ "step": 145000
2040
+ },
2041
+ {
2042
+ "epoch": 30.477586929199834,
2043
+ "grad_norm": 4.879722595214844,
2044
+ "learning_rate": 1.9535609551738584e-05,
2045
+ "loss": 5.9577,
2046
+ "step": 145500
2047
+ },
2048
+ {
2049
+ "epoch": 30.58232090490155,
2050
+ "grad_norm": 5.470447540283203,
2051
+ "learning_rate": 1.9430875576036867e-05,
2052
+ "loss": 5.9672,
2053
+ "step": 146000
2054
+ },
2055
+ {
2056
+ "epoch": 30.687054880603267,
2057
+ "grad_norm": 5.818385124206543,
2058
+ "learning_rate": 1.932614160033515e-05,
2059
+ "loss": 5.9501,
2060
+ "step": 146500
2061
+ },
2062
+ {
2063
+ "epoch": 30.791788856304986,
2064
+ "grad_norm": 5.907487392425537,
2065
+ "learning_rate": 1.9221617092584834e-05,
2066
+ "loss": 5.9458,
2067
+ "step": 147000
2068
+ },
2069
+ {
2070
+ "epoch": 30.896522832006703,
2071
+ "grad_norm": 4.739224433898926,
2072
+ "learning_rate": 1.911688311688312e-05,
2073
+ "loss": 5.935,
2074
+ "step": 147500
2075
+ },
2076
+ {
2077
+ "epoch": 31.00125680770842,
2078
+ "grad_norm": 4.57131814956665,
2079
+ "learning_rate": 1.90121491411814e-05,
2080
+ "loss": 5.945,
2081
+ "step": 148000
2082
+ },
2083
+ {
2084
+ "epoch": 31.10599078341014,
2085
+ "grad_norm": 5.128586769104004,
2086
+ "learning_rate": 1.8907415165479683e-05,
2087
+ "loss": 5.9494,
2088
+ "step": 148500
2089
+ },
2090
+ {
2091
+ "epoch": 31.210724759111855,
2092
+ "grad_norm": 4.871676921844482,
2093
+ "learning_rate": 1.880289065772937e-05,
2094
+ "loss": 5.9415,
2095
+ "step": 149000
2096
+ },
2097
+ {
2098
+ "epoch": 31.315458734813575,
2099
+ "grad_norm": 5.380068778991699,
2100
+ "learning_rate": 1.8698156682027652e-05,
2101
+ "loss": 5.939,
2102
+ "step": 149500
2103
+ },
2104
+ {
2105
+ "epoch": 31.42019271051529,
2106
+ "grad_norm": 5.430812835693359,
2107
+ "learning_rate": 1.8593422706325932e-05,
2108
+ "loss": 5.9276,
2109
+ "step": 150000
2110
+ },
2111
+ {
2112
+ "epoch": 31.524926686217007,
2113
+ "grad_norm": 4.7710442543029785,
2114
+ "learning_rate": 1.8488688730624215e-05,
2115
+ "loss": 5.9413,
2116
+ "step": 150500
2117
+ },
2118
+ {
2119
+ "epoch": 31.629660661918727,
2120
+ "grad_norm": 5.183919906616211,
2121
+ "learning_rate": 1.8383954754922498e-05,
2122
+ "loss": 5.9257,
2123
+ "step": 151000
2124
+ },
2125
+ {
2126
+ "epoch": 31.734394637620444,
2127
+ "grad_norm": 4.851598739624023,
2128
+ "learning_rate": 1.827943024717218e-05,
2129
+ "loss": 5.9251,
2130
+ "step": 151500
2131
+ },
2132
+ {
2133
+ "epoch": 31.839128613322163,
2134
+ "grad_norm": 4.835882663726807,
2135
+ "learning_rate": 1.8174696271470464e-05,
2136
+ "loss": 5.92,
2137
+ "step": 152000
2138
+ },
2139
+ {
2140
+ "epoch": 31.94386258902388,
2141
+ "grad_norm": 5.428823947906494,
2142
+ "learning_rate": 1.8069962295768748e-05,
2143
+ "loss": 5.9146,
2144
+ "step": 152500
2145
+ },
2146
+ {
2147
+ "epoch": 32.0485965647256,
2148
+ "grad_norm": 6.11329984664917,
2149
+ "learning_rate": 1.796522832006703e-05,
2150
+ "loss": 5.9179,
2151
+ "step": 153000
2152
+ },
2153
+ {
2154
+ "epoch": 32.15333054042731,
2155
+ "grad_norm": 4.836859226226807,
2156
+ "learning_rate": 1.7860703812316717e-05,
2157
+ "loss": 5.9189,
2158
+ "step": 153500
2159
+ },
2160
+ {
2161
+ "epoch": 32.25806451612903,
2162
+ "grad_norm": 4.598475456237793,
2163
+ "learning_rate": 1.7755969836615e-05,
2164
+ "loss": 5.9207,
2165
+ "step": 154000
2166
+ },
2167
+ {
2168
+ "epoch": 32.36279849183075,
2169
+ "grad_norm": 4.638394832611084,
2170
+ "learning_rate": 1.765123586091328e-05,
2171
+ "loss": 5.915,
2172
+ "step": 154500
2173
+ },
2174
+ {
2175
+ "epoch": 32.467532467532465,
2176
+ "grad_norm": 5.637279987335205,
2177
+ "learning_rate": 1.7546501885211563e-05,
2178
+ "loss": 5.9228,
2179
+ "step": 155000
2180
+ },
2181
+ {
2182
+ "epoch": 32.572266443234184,
2183
+ "grad_norm": 4.516068458557129,
2184
+ "learning_rate": 1.7441767909509846e-05,
2185
+ "loss": 5.9322,
2186
+ "step": 155500
2187
+ },
2188
+ {
2189
+ "epoch": 32.677000418935904,
2190
+ "grad_norm": 4.652084827423096,
2191
+ "learning_rate": 1.7337243401759533e-05,
2192
+ "loss": 5.9395,
2193
+ "step": 156000
2194
+ },
2195
+ {
2196
+ "epoch": 32.78173439463762,
2197
+ "grad_norm": 5.667607307434082,
2198
+ "learning_rate": 1.7232509426057812e-05,
2199
+ "loss": 5.9172,
2200
+ "step": 156500
2201
+ },
2202
+ {
2203
+ "epoch": 32.88646837033934,
2204
+ "grad_norm": 4.980391025543213,
2205
+ "learning_rate": 1.7127775450356095e-05,
2206
+ "loss": 5.934,
2207
+ "step": 157000
2208
+ },
2209
+ {
2210
+ "epoch": 32.99120234604106,
2211
+ "grad_norm": 4.39646053314209,
2212
+ "learning_rate": 1.702304147465438e-05,
2213
+ "loss": 5.914,
2214
+ "step": 157500
2215
+ },
2216
+ {
2217
+ "epoch": 33.095936321742776,
2218
+ "grad_norm": 5.263533115386963,
2219
+ "learning_rate": 1.6918516966904062e-05,
2220
+ "loss": 5.9094,
2221
+ "step": 158000
2222
+ },
2223
+ {
2224
+ "epoch": 33.20067029744449,
2225
+ "grad_norm": 4.661126136779785,
2226
+ "learning_rate": 1.6813782991202348e-05,
2227
+ "loss": 5.9189,
2228
+ "step": 158500
2229
+ },
2230
+ {
2231
+ "epoch": 33.30540427314621,
2232
+ "grad_norm": 4.935306549072266,
2233
+ "learning_rate": 1.670904901550063e-05,
2234
+ "loss": 5.8944,
2235
+ "step": 159000
2236
+ },
2237
+ {
2238
+ "epoch": 33.41013824884793,
2239
+ "grad_norm": 5.82065486907959,
2240
+ "learning_rate": 1.660431503979891e-05,
2241
+ "loss": 5.892,
2242
+ "step": 159500
2243
+ },
2244
+ {
2245
+ "epoch": 33.51487222454964,
2246
+ "grad_norm": 4.6220927238464355,
2247
+ "learning_rate": 1.6499581064097194e-05,
2248
+ "loss": 5.9022,
2249
+ "step": 160000
2250
+ },
2251
+ {
2252
+ "epoch": 33.61960620025136,
2253
+ "grad_norm": 5.109046936035156,
2254
+ "learning_rate": 1.639505655634688e-05,
2255
+ "loss": 5.8933,
2256
+ "step": 160500
2257
+ },
2258
+ {
2259
+ "epoch": 33.72434017595308,
2260
+ "grad_norm": 5.230437278747559,
2261
+ "learning_rate": 1.6290322580645164e-05,
2262
+ "loss": 5.9107,
2263
+ "step": 161000
2264
+ },
2265
+ {
2266
+ "epoch": 33.829074151654794,
2267
+ "grad_norm": 6.466080188751221,
2268
+ "learning_rate": 1.6185588604943443e-05,
2269
+ "loss": 5.9077,
2270
+ "step": 161500
2271
+ },
2272
+ {
2273
+ "epoch": 33.933808127356514,
2274
+ "grad_norm": 4.655428409576416,
2275
+ "learning_rate": 1.6080854629241726e-05,
2276
+ "loss": 5.9186,
2277
+ "step": 162000
2278
+ },
2279
+ {
2280
+ "epoch": 34.038542103058234,
2281
+ "grad_norm": 6.136354923248291,
2282
+ "learning_rate": 1.597612065354001e-05,
2283
+ "loss": 5.8815,
2284
+ "step": 162500
2285
+ },
2286
+ {
2287
+ "epoch": 34.143276078759946,
2288
+ "grad_norm": 5.668376445770264,
2289
+ "learning_rate": 1.587138667783829e-05,
2290
+ "loss": 5.8872,
2291
+ "step": 163000
2292
+ },
2293
+ {
2294
+ "epoch": 34.248010054461666,
2295
+ "grad_norm": 5.579314708709717,
2296
+ "learning_rate": 1.5766862170087976e-05,
2297
+ "loss": 5.8966,
2298
+ "step": 163500
2299
+ },
2300
+ {
2301
+ "epoch": 34.352744030163386,
2302
+ "grad_norm": 5.474893569946289,
2303
+ "learning_rate": 1.5662128194386262e-05,
2304
+ "loss": 5.8865,
2305
+ "step": 164000
2306
+ },
2307
+ {
2308
+ "epoch": 34.457478005865106,
2309
+ "grad_norm": 4.853377342224121,
2310
+ "learning_rate": 1.5557394218684542e-05,
2311
+ "loss": 5.8718,
2312
+ "step": 164500
2313
+ },
2314
+ {
2315
+ "epoch": 34.56221198156682,
2316
+ "grad_norm": 4.8684563636779785,
2317
+ "learning_rate": 1.5452660242982825e-05,
2318
+ "loss": 5.8896,
2319
+ "step": 165000
2320
+ },
2321
+ {
2322
+ "epoch": 34.66694595726854,
2323
+ "grad_norm": 4.851233959197998,
2324
+ "learning_rate": 1.5347926267281108e-05,
2325
+ "loss": 5.8908,
2326
+ "step": 165500
2327
+ },
2328
+ {
2329
+ "epoch": 34.77167993297026,
2330
+ "grad_norm": 6.647628307342529,
2331
+ "learning_rate": 1.524319229157939e-05,
2332
+ "loss": 5.8972,
2333
+ "step": 166000
2334
+ },
2335
+ {
2336
+ "epoch": 34.87641390867197,
2337
+ "grad_norm": 5.826745986938477,
2338
+ "learning_rate": 1.5138667783829074e-05,
2339
+ "loss": 5.8852,
2340
+ "step": 166500
2341
+ },
2342
+ {
2343
+ "epoch": 34.98114788437369,
2344
+ "grad_norm": 5.109675407409668,
2345
+ "learning_rate": 1.5033933808127357e-05,
2346
+ "loss": 5.8971,
2347
+ "step": 167000
2348
+ },
2349
+ {
2350
+ "epoch": 35.08588186007541,
2351
+ "grad_norm": 5.743017196655273,
2352
+ "learning_rate": 1.4929199832425639e-05,
2353
+ "loss": 5.8892,
2354
+ "step": 167500
2355
+ },
2356
+ {
2357
+ "epoch": 35.19061583577712,
2358
+ "grad_norm": 4.862270355224609,
2359
+ "learning_rate": 1.482446585672392e-05,
2360
+ "loss": 5.8976,
2361
+ "step": 168000
2362
+ },
2363
+ {
2364
+ "epoch": 35.29534981147884,
2365
+ "grad_norm": 5.532686233520508,
2366
+ "learning_rate": 1.4719941348973607e-05,
2367
+ "loss": 5.8716,
2368
+ "step": 168500
2369
+ },
2370
+ {
2371
+ "epoch": 35.40008378718056,
2372
+ "grad_norm": 5.498019695281982,
2373
+ "learning_rate": 1.4615207373271891e-05,
2374
+ "loss": 5.8768,
2375
+ "step": 169000
2376
+ },
2377
+ {
2378
+ "epoch": 35.504817762882276,
2379
+ "grad_norm": 4.7324042320251465,
2380
+ "learning_rate": 1.4510473397570173e-05,
2381
+ "loss": 5.8739,
2382
+ "step": 169500
2383
+ },
2384
+ {
2385
+ "epoch": 35.609551738583995,
2386
+ "grad_norm": 4.973413944244385,
2387
+ "learning_rate": 1.4405739421868456e-05,
2388
+ "loss": 5.8801,
2389
+ "step": 170000
2390
+ },
2391
+ {
2392
+ "epoch": 35.714285714285715,
2393
+ "grad_norm": 4.977658271789551,
2394
+ "learning_rate": 1.430121491411814e-05,
2395
+ "loss": 5.8644,
2396
+ "step": 170500
2397
+ },
2398
+ {
2399
+ "epoch": 35.819019689987435,
2400
+ "grad_norm": 5.551715850830078,
2401
+ "learning_rate": 1.4196480938416424e-05,
2402
+ "loss": 5.8789,
2403
+ "step": 171000
2404
+ },
2405
+ {
2406
+ "epoch": 35.92375366568915,
2407
+ "grad_norm": 5.135740756988525,
2408
+ "learning_rate": 1.4091746962714705e-05,
2409
+ "loss": 5.8862,
2410
+ "step": 171500
2411
+ },
2412
+ {
2413
+ "epoch": 36.02848764139087,
2414
+ "grad_norm": 5.068655967712402,
2415
+ "learning_rate": 1.3987012987012987e-05,
2416
+ "loss": 5.879,
2417
+ "step": 172000
2418
+ },
2419
+ {
2420
+ "epoch": 36.13322161709259,
2421
+ "grad_norm": 5.393857479095459,
2422
+ "learning_rate": 1.388227901131127e-05,
2423
+ "loss": 5.8544,
2424
+ "step": 172500
2425
+ },
2426
+ {
2427
+ "epoch": 36.2379555927943,
2428
+ "grad_norm": 5.854538440704346,
2429
+ "learning_rate": 1.3777545035609551e-05,
2430
+ "loss": 5.8824,
2431
+ "step": 173000
2432
+ },
2433
+ {
2434
+ "epoch": 36.34268956849602,
2435
+ "grad_norm": 5.566401481628418,
2436
+ "learning_rate": 1.3673020527859238e-05,
2437
+ "loss": 5.8587,
2438
+ "step": 173500
2439
+ },
2440
+ {
2441
+ "epoch": 36.44742354419774,
2442
+ "grad_norm": 6.091250896453857,
2443
+ "learning_rate": 1.3568286552157519e-05,
2444
+ "loss": 5.8624,
2445
+ "step": 174000
2446
+ },
2447
+ {
2448
+ "epoch": 36.55215751989945,
2449
+ "grad_norm": 4.826417922973633,
2450
+ "learning_rate": 1.3463552576455804e-05,
2451
+ "loss": 5.879,
2452
+ "step": 174500
2453
+ },
2454
+ {
2455
+ "epoch": 36.65689149560117,
2456
+ "grad_norm": 5.28770637512207,
2457
+ "learning_rate": 1.3358818600754087e-05,
2458
+ "loss": 5.8632,
2459
+ "step": 175000
2460
+ },
2461
+ {
2462
+ "epoch": 36.76162547130289,
2463
+ "grad_norm": 5.072086811065674,
2464
+ "learning_rate": 1.3254084625052368e-05,
2465
+ "loss": 5.8698,
2466
+ "step": 175500
2467
+ },
2468
+ {
2469
+ "epoch": 36.866359447004605,
2470
+ "grad_norm": 6.194067001342773,
2471
+ "learning_rate": 1.3149560117302053e-05,
2472
+ "loss": 5.8701,
2473
+ "step": 176000
2474
+ },
2475
+ {
2476
+ "epoch": 36.971093422706325,
2477
+ "grad_norm": 5.250491142272949,
2478
+ "learning_rate": 1.3044826141600336e-05,
2479
+ "loss": 5.855,
2480
+ "step": 176500
2481
+ },
2482
+ {
2483
+ "epoch": 37.075827398408045,
2484
+ "grad_norm": 4.9726080894470215,
2485
+ "learning_rate": 1.2940092165898618e-05,
2486
+ "loss": 5.8613,
2487
+ "step": 177000
2488
+ },
2489
+ {
2490
+ "epoch": 37.180561374109764,
2491
+ "grad_norm": 5.526548385620117,
2492
+ "learning_rate": 1.28353581901969e-05,
2493
+ "loss": 5.8519,
2494
+ "step": 177500
2495
+ },
2496
+ {
2497
+ "epoch": 37.28529534981148,
2498
+ "grad_norm": 5.5989861488342285,
2499
+ "learning_rate": 1.2730624214495182e-05,
2500
+ "loss": 5.8642,
2501
+ "step": 178000
2502
+ },
2503
+ {
2504
+ "epoch": 37.3900293255132,
2505
+ "grad_norm": 5.138686180114746,
2506
+ "learning_rate": 1.2625890238793465e-05,
2507
+ "loss": 5.852,
2508
+ "step": 178500
2509
+ },
2510
+ {
2511
+ "epoch": 37.49476330121492,
2512
+ "grad_norm": 5.0514326095581055,
2513
+ "learning_rate": 1.252115626309175e-05,
2514
+ "loss": 5.8484,
2515
+ "step": 179000
2516
+ },
2517
+ {
2518
+ "epoch": 37.59949727691663,
2519
+ "grad_norm": 4.9300360679626465,
2520
+ "learning_rate": 1.241642228739003e-05,
2521
+ "loss": 5.849,
2522
+ "step": 179500
2523
+ },
2524
+ {
2525
+ "epoch": 37.70423125261835,
2526
+ "grad_norm": 5.487224102020264,
2527
+ "learning_rate": 1.2311897779639716e-05,
2528
+ "loss": 5.8562,
2529
+ "step": 180000
2530
+ },
2531
+ {
2532
+ "epoch": 37.80896522832007,
2533
+ "grad_norm": 5.826539516448975,
2534
+ "learning_rate": 1.2207163803937999e-05,
2535
+ "loss": 5.8665,
2536
+ "step": 180500
2537
+ },
2538
+ {
2539
+ "epoch": 37.91369920402178,
2540
+ "grad_norm": 5.733819961547852,
2541
+ "learning_rate": 1.2102639296187684e-05,
2542
+ "loss": 5.8569,
2543
+ "step": 181000
2544
+ },
2545
+ {
2546
+ "epoch": 38.0184331797235,
2547
+ "grad_norm": 4.917960166931152,
2548
+ "learning_rate": 1.1997905320485967e-05,
2549
+ "loss": 5.8395,
2550
+ "step": 181500
2551
+ },
2552
+ {
2553
+ "epoch": 38.12316715542522,
2554
+ "grad_norm": 5.337119102478027,
2555
+ "learning_rate": 1.1893171344784248e-05,
2556
+ "loss": 5.854,
2557
+ "step": 182000
2558
+ },
2559
+ {
2560
+ "epoch": 38.227901131126934,
2561
+ "grad_norm": 5.299139022827148,
2562
+ "learning_rate": 1.178843736908253e-05,
2563
+ "loss": 5.8433,
2564
+ "step": 182500
2565
+ },
2566
+ {
2567
+ "epoch": 38.332635106828654,
2568
+ "grad_norm": 5.900153160095215,
2569
+ "learning_rate": 1.1683703393380813e-05,
2570
+ "loss": 5.8535,
2571
+ "step": 183000
2572
+ },
2573
+ {
2574
+ "epoch": 38.437369082530374,
2575
+ "grad_norm": 6.776584625244141,
2576
+ "learning_rate": 1.15791788856305e-05,
2577
+ "loss": 5.8454,
2578
+ "step": 183500
2579
+ },
2580
+ {
2581
+ "epoch": 38.542103058232094,
2582
+ "grad_norm": 6.258368015289307,
2583
+ "learning_rate": 1.1474444909928783e-05,
2584
+ "loss": 5.8354,
2585
+ "step": 184000
2586
+ },
2587
+ {
2588
+ "epoch": 38.64683703393381,
2589
+ "grad_norm": 5.288670539855957,
2590
+ "learning_rate": 1.1369710934227064e-05,
2591
+ "loss": 5.8458,
2592
+ "step": 184500
2593
+ },
2594
+ {
2595
+ "epoch": 38.751571009635526,
2596
+ "grad_norm": 5.596650123596191,
2597
+ "learning_rate": 1.1264976958525345e-05,
2598
+ "loss": 5.8387,
2599
+ "step": 185000
2600
+ },
2601
+ {
2602
+ "epoch": 38.856304985337246,
2603
+ "grad_norm": 5.121638774871826,
2604
+ "learning_rate": 1.1160452450775032e-05,
2605
+ "loss": 5.8268,
2606
+ "step": 185500
2607
+ },
2608
+ {
2609
+ "epoch": 38.96103896103896,
2610
+ "grad_norm": 4.5758442878723145,
2611
+ "learning_rate": 1.1055718475073313e-05,
2612
+ "loss": 5.83,
2613
+ "step": 186000
2614
+ },
2615
+ {
2616
+ "epoch": 39.06577293674068,
2617
+ "grad_norm": 5.161282539367676,
2618
+ "learning_rate": 1.0950984499371596e-05,
2619
+ "loss": 5.8544,
2620
+ "step": 186500
2621
+ },
2622
+ {
2623
+ "epoch": 39.1705069124424,
2624
+ "grad_norm": 4.628884315490723,
2625
+ "learning_rate": 1.084625052366988e-05,
2626
+ "loss": 5.8474,
2627
+ "step": 187000
2628
+ },
2629
+ {
2630
+ "epoch": 39.27524088814411,
2631
+ "grad_norm": 5.854598045349121,
2632
+ "learning_rate": 1.074151654796816e-05,
2633
+ "loss": 5.8501,
2634
+ "step": 187500
2635
+ },
2636
+ {
2637
+ "epoch": 39.37997486384583,
2638
+ "grad_norm": 5.315525054931641,
2639
+ "learning_rate": 1.0636782572266444e-05,
2640
+ "loss": 5.8265,
2641
+ "step": 188000
2642
+ },
2643
+ {
2644
+ "epoch": 39.48470883954755,
2645
+ "grad_norm": 6.078185081481934,
2646
+ "learning_rate": 1.0532048596564727e-05,
2647
+ "loss": 5.8419,
2648
+ "step": 188500
2649
+ },
2650
+ {
2651
+ "epoch": 39.589442815249264,
2652
+ "grad_norm": 5.223086357116699,
2653
+ "learning_rate": 1.0427524088814412e-05,
2654
+ "loss": 5.8197,
2655
+ "step": 189000
2656
+ },
2657
+ {
2658
+ "epoch": 39.69417679095098,
2659
+ "grad_norm": 5.235757827758789,
2660
+ "learning_rate": 1.0322790113112695e-05,
2661
+ "loss": 5.8245,
2662
+ "step": 189500
2663
+ },
2664
+ {
2665
+ "epoch": 39.7989107666527,
2666
+ "grad_norm": 5.124643325805664,
2667
+ "learning_rate": 1.0218056137410976e-05,
2668
+ "loss": 5.839,
2669
+ "step": 190000
2670
+ },
2671
+ {
2672
+ "epoch": 39.90364474235442,
2673
+ "grad_norm": 5.613321304321289,
2674
+ "learning_rate": 1.011332216170926e-05,
2675
+ "loss": 5.844,
2676
+ "step": 190500
2677
+ },
2678
+ {
2679
+ "epoch": 40.008378718056136,
2680
+ "grad_norm": 5.873430252075195,
2681
+ "learning_rate": 1.0008588186007542e-05,
2682
+ "loss": 5.837,
2683
+ "step": 191000
2684
+ },
2685
+ {
2686
+ "epoch": 40.113112693757856,
2687
+ "grad_norm": 5.089309215545654,
2688
+ "learning_rate": 9.903854210305824e-06,
2689
+ "loss": 5.8384,
2690
+ "step": 191500
2691
+ },
2692
+ {
2693
+ "epoch": 40.217846669459576,
2694
+ "grad_norm": 7.3569817543029785,
2695
+ "learning_rate": 9.79932970255551e-06,
2696
+ "loss": 5.8232,
2697
+ "step": 192000
2698
+ },
2699
+ {
2700
+ "epoch": 40.32258064516129,
2701
+ "grad_norm": 6.024489402770996,
2702
+ "learning_rate": 9.694595726853792e-06,
2703
+ "loss": 5.8292,
2704
+ "step": 192500
2705
+ },
2706
+ {
2707
+ "epoch": 40.42731462086301,
2708
+ "grad_norm": 5.7150983810424805,
2709
+ "learning_rate": 9.589861751152073e-06,
2710
+ "loss": 5.8614,
2711
+ "step": 193000
2712
+ },
2713
+ {
2714
+ "epoch": 40.53204859656473,
2715
+ "grad_norm": 4.717107772827148,
2716
+ "learning_rate": 9.485127775450356e-06,
2717
+ "loss": 5.8092,
2718
+ "step": 193500
2719
+ },
2720
+ {
2721
+ "epoch": 40.63678257226644,
2722
+ "grad_norm": 4.9722490310668945,
2723
+ "learning_rate": 9.380603267700043e-06,
2724
+ "loss": 5.8231,
2725
+ "step": 194000
2726
+ },
2727
+ {
2728
+ "epoch": 40.74151654796816,
2729
+ "grad_norm": 5.593094825744629,
2730
+ "learning_rate": 9.275869291998326e-06,
2731
+ "loss": 5.8339,
2732
+ "step": 194500
2733
+ },
2734
+ {
2735
+ "epoch": 40.84625052366988,
2736
+ "grad_norm": 5.731310844421387,
2737
+ "learning_rate": 9.171135316296607e-06,
2738
+ "loss": 5.8381,
2739
+ "step": 195000
2740
+ },
2741
+ {
2742
+ "epoch": 40.95098449937159,
2743
+ "grad_norm": 5.072065353393555,
2744
+ "learning_rate": 9.066401340594889e-06,
2745
+ "loss": 5.8367,
2746
+ "step": 195500
2747
+ },
2748
+ {
2749
+ "epoch": 41.05571847507331,
2750
+ "grad_norm": 5.219040870666504,
2751
+ "learning_rate": 8.961667364893172e-06,
2752
+ "loss": 5.8234,
2753
+ "step": 196000
2754
+ },
2755
+ {
2756
+ "epoch": 41.16045245077503,
2757
+ "grad_norm": 5.844238758087158,
2758
+ "learning_rate": 8.856933389191455e-06,
2759
+ "loss": 5.8347,
2760
+ "step": 196500
2761
+ },
2762
+ {
2763
+ "epoch": 41.26518642647675,
2764
+ "grad_norm": 6.088447093963623,
2765
+ "learning_rate": 8.752199413489736e-06,
2766
+ "loss": 5.8178,
2767
+ "step": 197000
2768
+ },
2769
+ {
2770
+ "epoch": 41.369920402178465,
2771
+ "grad_norm": 5.14108943939209,
2772
+ "learning_rate": 8.647465437788019e-06,
2773
+ "loss": 5.8248,
2774
+ "step": 197500
2775
+ },
2776
+ {
2777
+ "epoch": 41.474654377880185,
2778
+ "grad_norm": 5.424249172210693,
2779
+ "learning_rate": 8.542940930037704e-06,
2780
+ "loss": 5.8113,
2781
+ "step": 198000
2782
+ },
2783
+ {
2784
+ "epoch": 41.579388353581905,
2785
+ "grad_norm": 4.888121604919434,
2786
+ "learning_rate": 8.43841642228739e-06,
2787
+ "loss": 5.8111,
2788
+ "step": 198500
2789
+ },
2790
+ {
2791
+ "epoch": 41.68412232928362,
2792
+ "grad_norm": 4.9909515380859375,
2793
+ "learning_rate": 8.333682446585672e-06,
2794
+ "loss": 5.8276,
2795
+ "step": 199000
2796
+ },
2797
+ {
2798
+ "epoch": 41.78885630498534,
2799
+ "grad_norm": 5.032175540924072,
2800
+ "learning_rate": 8.228948470883955e-06,
2801
+ "loss": 5.8332,
2802
+ "step": 199500
2803
+ },
2804
+ {
2805
+ "epoch": 41.89359028068706,
2806
+ "grad_norm": 5.116880416870117,
2807
+ "learning_rate": 8.124214495182238e-06,
2808
+ "loss": 5.8233,
2809
+ "step": 200000
2810
+ },
2811
+ {
2812
+ "epoch": 41.99832425638877,
2813
+ "grad_norm": 5.235647678375244,
2814
+ "learning_rate": 8.019689987431923e-06,
2815
+ "loss": 5.8297,
2816
+ "step": 200500
2817
+ },
2818
+ {
2819
+ "epoch": 42.10305823209049,
2820
+ "grad_norm": 5.445380210876465,
2821
+ "learning_rate": 7.914956011730206e-06,
2822
+ "loss": 5.8159,
2823
+ "step": 201000
2824
+ },
2825
+ {
2826
+ "epoch": 42.20779220779221,
2827
+ "grad_norm": 4.979036331176758,
2828
+ "learning_rate": 7.810222036028488e-06,
2829
+ "loss": 5.809,
2830
+ "step": 201500
2831
+ },
2832
+ {
2833
+ "epoch": 42.31252618349392,
2834
+ "grad_norm": 5.359362602233887,
2835
+ "learning_rate": 7.70548806032677e-06,
2836
+ "loss": 5.8346,
2837
+ "step": 202000
2838
+ },
2839
+ {
2840
+ "epoch": 42.41726015919564,
2841
+ "grad_norm": 5.264519214630127,
2842
+ "learning_rate": 7.600754084625053e-06,
2843
+ "loss": 5.8089,
2844
+ "step": 202500
2845
+ },
2846
+ {
2847
+ "epoch": 42.52199413489736,
2848
+ "grad_norm": 5.985982894897461,
2849
+ "learning_rate": 7.496020108923335e-06,
2850
+ "loss": 5.8192,
2851
+ "step": 203000
2852
+ },
2853
+ {
2854
+ "epoch": 42.626728110599075,
2855
+ "grad_norm": 5.505626201629639,
2856
+ "learning_rate": 7.391286133221617e-06,
2857
+ "loss": 5.8095,
2858
+ "step": 203500
2859
+ },
2860
+ {
2861
+ "epoch": 42.731462086300795,
2862
+ "grad_norm": 5.069738388061523,
2863
+ "learning_rate": 7.286552157519899e-06,
2864
+ "loss": 5.8186,
2865
+ "step": 204000
2866
+ },
2867
+ {
2868
+ "epoch": 42.836196062002514,
2869
+ "grad_norm": 6.004745960235596,
2870
+ "learning_rate": 7.182027649769586e-06,
2871
+ "loss": 5.8136,
2872
+ "step": 204500
2873
+ },
2874
+ {
2875
+ "epoch": 42.940930037704234,
2876
+ "grad_norm": 6.299502372741699,
2877
+ "learning_rate": 7.077293674067868e-06,
2878
+ "loss": 5.8213,
2879
+ "step": 205000
2880
+ },
2881
+ {
2882
+ "epoch": 43.04566401340595,
2883
+ "grad_norm": 6.302718162536621,
2884
+ "learning_rate": 6.97255969836615e-06,
2885
+ "loss": 5.8075,
2886
+ "step": 205500
2887
+ },
2888
+ {
2889
+ "epoch": 43.15039798910767,
2890
+ "grad_norm": 5.921250343322754,
2891
+ "learning_rate": 6.867825722664433e-06,
2892
+ "loss": 5.8276,
2893
+ "step": 206000
2894
+ },
2895
+ {
2896
+ "epoch": 43.25513196480939,
2897
+ "grad_norm": 5.123110771179199,
2898
+ "learning_rate": 6.763091746962715e-06,
2899
+ "loss": 5.7965,
2900
+ "step": 206500
2901
+ },
2902
+ {
2903
+ "epoch": 43.3598659405111,
2904
+ "grad_norm": 5.187294006347656,
2905
+ "learning_rate": 6.658357771260998e-06,
2906
+ "loss": 5.8137,
2907
+ "step": 207000
2908
+ },
2909
+ {
2910
+ "epoch": 43.46459991621282,
2911
+ "grad_norm": 5.407510757446289,
2912
+ "learning_rate": 6.55362379555928e-06,
2913
+ "loss": 5.8305,
2914
+ "step": 207500
2915
+ },
2916
+ {
2917
+ "epoch": 43.56933389191454,
2918
+ "grad_norm": 5.892600059509277,
2919
+ "learning_rate": 6.449099287808966e-06,
2920
+ "loss": 5.8167,
2921
+ "step": 208000
2922
+ },
2923
+ {
2924
+ "epoch": 43.67406786761625,
2925
+ "grad_norm": 5.39382266998291,
2926
+ "learning_rate": 6.344365312107248e-06,
2927
+ "loss": 5.8185,
2928
+ "step": 208500
2929
+ },
2930
+ {
2931
+ "epoch": 43.77880184331797,
2932
+ "grad_norm": 5.608034133911133,
2933
+ "learning_rate": 6.23963133640553e-06,
2934
+ "loss": 5.8051,
2935
+ "step": 209000
2936
+ },
2937
+ {
2938
+ "epoch": 43.88353581901969,
2939
+ "grad_norm": 6.069722652435303,
2940
+ "learning_rate": 6.1348973607038125e-06,
2941
+ "loss": 5.8101,
2942
+ "step": 209500
2943
+ },
2944
+ {
2945
+ "epoch": 43.988269794721404,
2946
+ "grad_norm": 5.938599109649658,
2947
+ "learning_rate": 6.0301633850020955e-06,
2948
+ "loss": 5.807,
2949
+ "step": 210000
2950
+ },
2951
+ {
2952
+ "epoch": 44.093003770423124,
2953
+ "grad_norm": 5.808456897735596,
2954
+ "learning_rate": 5.925429409300378e-06,
2955
+ "loss": 5.8246,
2956
+ "step": 210500
2957
+ },
2958
+ {
2959
+ "epoch": 44.197737746124844,
2960
+ "grad_norm": 5.229996681213379,
2961
+ "learning_rate": 5.820904901550063e-06,
2962
+ "loss": 5.8029,
2963
+ "step": 211000
2964
+ },
2965
+ {
2966
+ "epoch": 44.302471721826564,
2967
+ "grad_norm": 5.295706748962402,
2968
+ "learning_rate": 5.716170925848346e-06,
2969
+ "loss": 5.8094,
2970
+ "step": 211500
2971
+ },
2972
+ {
2973
+ "epoch": 44.407205697528276,
2974
+ "grad_norm": 5.649194240570068,
2975
+ "learning_rate": 5.611436950146628e-06,
2976
+ "loss": 5.811,
2977
+ "step": 212000
2978
+ },
2979
+ {
2980
+ "epoch": 44.511939673229996,
2981
+ "grad_norm": 6.5928521156311035,
2982
+ "learning_rate": 5.50670297444491e-06,
2983
+ "loss": 5.7974,
2984
+ "step": 212500
2985
+ },
2986
+ {
2987
+ "epoch": 44.616673648931716,
2988
+ "grad_norm": 6.246605396270752,
2989
+ "learning_rate": 5.401968998743192e-06,
2990
+ "loss": 5.8011,
2991
+ "step": 213000
2992
+ },
2993
+ {
2994
+ "epoch": 44.72140762463343,
2995
+ "grad_norm": 5.312093734741211,
2996
+ "learning_rate": 5.2972350230414745e-06,
2997
+ "loss": 5.7819,
2998
+ "step": 213500
2999
+ },
3000
+ {
3001
+ "epoch": 44.82614160033515,
3002
+ "grad_norm": 5.348554611206055,
3003
+ "learning_rate": 5.19271051529116e-06,
3004
+ "loss": 5.8027,
3005
+ "step": 214000
3006
+ },
3007
+ {
3008
+ "epoch": 44.93087557603687,
3009
+ "grad_norm": 5.95352029800415,
3010
+ "learning_rate": 5.087976539589443e-06,
3011
+ "loss": 5.8046,
3012
+ "step": 214500
3013
+ },
3014
+ {
3015
+ "epoch": 45.03560955173858,
3016
+ "grad_norm": 5.978014945983887,
3017
+ "learning_rate": 4.983242563887726e-06,
3018
+ "loss": 5.8021,
3019
+ "step": 215000
3020
+ },
3021
+ {
3022
+ "epoch": 45.1403435274403,
3023
+ "grad_norm": 5.595849990844727,
3024
+ "learning_rate": 4.878508588186008e-06,
3025
+ "loss": 5.7996,
3026
+ "step": 215500
3027
+ },
3028
+ {
3029
+ "epoch": 45.24507750314202,
3030
+ "grad_norm": 5.570345401763916,
3031
+ "learning_rate": 4.77377461248429e-06,
3032
+ "loss": 5.7973,
3033
+ "step": 216000
3034
+ },
3035
+ {
3036
+ "epoch": 45.34981147884373,
3037
+ "grad_norm": 5.320748805999756,
3038
+ "learning_rate": 4.669040636782573e-06,
3039
+ "loss": 5.7886,
3040
+ "step": 216500
3041
+ },
3042
+ {
3043
+ "epoch": 45.45454545454545,
3044
+ "grad_norm": 4.676185607910156,
3045
+ "learning_rate": 4.564516129032258e-06,
3046
+ "loss": 5.7874,
3047
+ "step": 217000
3048
+ },
3049
+ {
3050
+ "epoch": 45.55927943024717,
3051
+ "grad_norm": 5.7768473625183105,
3052
+ "learning_rate": 4.45978215333054e-06,
3053
+ "loss": 5.7875,
3054
+ "step": 217500
3055
+ },
3056
+ {
3057
+ "epoch": 45.66401340594889,
3058
+ "grad_norm": 5.668895244598389,
3059
+ "learning_rate": 4.355048177628823e-06,
3060
+ "loss": 5.8121,
3061
+ "step": 218000
3062
+ },
3063
+ {
3064
+ "epoch": 45.768747381650606,
3065
+ "grad_norm": 5.033557891845703,
3066
+ "learning_rate": 4.2503142019271055e-06,
3067
+ "loss": 5.8137,
3068
+ "step": 218500
3069
+ },
3070
+ {
3071
+ "epoch": 45.873481357352325,
3072
+ "grad_norm": 6.15772819519043,
3073
+ "learning_rate": 4.145580226225388e-06,
3074
+ "loss": 5.8137,
3075
+ "step": 219000
3076
+ },
3077
+ {
3078
+ "epoch": 45.978215333054045,
3079
+ "grad_norm": 6.617910861968994,
3080
+ "learning_rate": 4.04084625052367e-06,
3081
+ "loss": 5.8079,
3082
+ "step": 219500
3083
+ },
3084
+ {
3085
+ "epoch": 46.08294930875576,
3086
+ "grad_norm": 5.210205554962158,
3087
+ "learning_rate": 3.936112274821952e-06,
3088
+ "loss": 5.8189,
3089
+ "step": 220000
3090
+ },
3091
+ {
3092
+ "epoch": 46.18768328445748,
3093
+ "grad_norm": 5.630945205688477,
3094
+ "learning_rate": 3.831587767071639e-06,
3095
+ "loss": 5.795,
3096
+ "step": 220500
3097
+ },
3098
+ {
3099
+ "epoch": 46.2924172601592,
3100
+ "grad_norm": 5.8690032958984375,
3101
+ "learning_rate": 3.7268537913699205e-06,
3102
+ "loss": 5.8019,
3103
+ "step": 221000
3104
+ },
3105
+ {
3106
+ "epoch": 46.39715123586091,
3107
+ "grad_norm": 5.787112712860107,
3108
+ "learning_rate": 3.6221198156682027e-06,
3109
+ "loss": 5.7901,
3110
+ "step": 221500
3111
+ },
3112
+ {
3113
+ "epoch": 46.50188521156263,
3114
+ "grad_norm": 5.568469524383545,
3115
+ "learning_rate": 3.5173858399664853e-06,
3116
+ "loss": 5.803,
3117
+ "step": 222000
3118
+ },
3119
+ {
3120
+ "epoch": 46.60661918726435,
3121
+ "grad_norm": 5.671326637268066,
3122
+ "learning_rate": 3.4126518642647675e-06,
3123
+ "loss": 5.7931,
3124
+ "step": 222500
3125
+ },
3126
+ {
3127
+ "epoch": 46.71135316296606,
3128
+ "grad_norm": 5.4085307121276855,
3129
+ "learning_rate": 3.30791788856305e-06,
3130
+ "loss": 5.7903,
3131
+ "step": 223000
3132
+ },
3133
+ {
3134
+ "epoch": 46.81608713866778,
3135
+ "grad_norm": 5.7440571784973145,
3136
+ "learning_rate": 3.203393380812736e-06,
3137
+ "loss": 5.7897,
3138
+ "step": 223500
3139
+ },
3140
+ {
3141
+ "epoch": 46.9208211143695,
3142
+ "grad_norm": 5.144542217254639,
3143
+ "learning_rate": 3.098659405111018e-06,
3144
+ "loss": 5.8145,
3145
+ "step": 224000
3146
+ },
3147
+ {
3148
+ "epoch": 47.02555509007122,
3149
+ "grad_norm": 5.842213153839111,
3150
+ "learning_rate": 2.9939254294093008e-06,
3151
+ "loss": 5.8046,
3152
+ "step": 224500
3153
+ },
3154
+ {
3155
+ "epoch": 47.130289065772935,
3156
+ "grad_norm": 6.161410331726074,
3157
+ "learning_rate": 2.8891914537075826e-06,
3158
+ "loss": 5.7965,
3159
+ "step": 225000
3160
+ },
3161
+ {
3162
+ "epoch": 47.235023041474655,
3163
+ "grad_norm": 6.173724174499512,
3164
+ "learning_rate": 2.784457478005865e-06,
3165
+ "loss": 5.7854,
3166
+ "step": 225500
3167
+ },
3168
+ {
3169
+ "epoch": 47.339757017176375,
3170
+ "grad_norm": 5.132796287536621,
3171
+ "learning_rate": 2.679932970255551e-06,
3172
+ "loss": 5.791,
3173
+ "step": 226000
3174
+ },
3175
+ {
3176
+ "epoch": 47.44449099287809,
3177
+ "grad_norm": 6.053417205810547,
3178
+ "learning_rate": 2.5751989945538332e-06,
3179
+ "loss": 5.7935,
3180
+ "step": 226500
3181
+ },
3182
+ {
3183
+ "epoch": 47.54922496857981,
3184
+ "grad_norm": 5.08466911315918,
3185
+ "learning_rate": 2.470465018852116e-06,
3186
+ "loss": 5.8006,
3187
+ "step": 227000
3188
+ },
3189
+ {
3190
+ "epoch": 47.65395894428153,
3191
+ "grad_norm": 6.060305595397949,
3192
+ "learning_rate": 2.365731043150398e-06,
3193
+ "loss": 5.7839,
3194
+ "step": 227500
3195
+ },
3196
+ {
3197
+ "epoch": 47.75869291998324,
3198
+ "grad_norm": 5.808520317077637,
3199
+ "learning_rate": 2.2609970674486806e-06,
3200
+ "loss": 5.7933,
3201
+ "step": 228000
3202
+ },
3203
+ {
3204
+ "epoch": 47.86342689568496,
3205
+ "grad_norm": 5.413971424102783,
3206
+ "learning_rate": 2.156472559698366e-06,
3207
+ "loss": 5.7985,
3208
+ "step": 228500
3209
+ },
3210
+ {
3211
+ "epoch": 47.96816087138668,
3212
+ "grad_norm": 6.4786529541015625,
3213
+ "learning_rate": 2.0517385839966487e-06,
3214
+ "loss": 5.8059,
3215
+ "step": 229000
3216
+ },
3217
+ {
3218
+ "epoch": 48.07289484708839,
3219
+ "grad_norm": 5.606147289276123,
3220
+ "learning_rate": 1.947004608294931e-06,
3221
+ "loss": 5.7934,
3222
+ "step": 229500
3223
+ },
3224
+ {
3225
+ "epoch": 48.17762882279011,
3226
+ "grad_norm": 5.827240467071533,
3227
+ "learning_rate": 1.8422706325932133e-06,
3228
+ "loss": 5.8007,
3229
+ "step": 230000
3230
+ },
3231
+ {
3232
+ "epoch": 48.28236279849183,
3233
+ "grad_norm": 5.678854465484619,
3234
+ "learning_rate": 1.7375366568914957e-06,
3235
+ "loss": 5.8038,
3236
+ "step": 230500
3237
+ },
3238
+ {
3239
+ "epoch": 48.38709677419355,
3240
+ "grad_norm": 5.42138671875,
3241
+ "learning_rate": 1.632802681189778e-06,
3242
+ "loss": 5.7976,
3243
+ "step": 231000
3244
+ },
3245
+ {
3246
+ "epoch": 48.491830749895264,
3247
+ "grad_norm": 6.03090238571167,
3248
+ "learning_rate": 1.5280687054880603e-06,
3249
+ "loss": 5.7874,
3250
+ "step": 231500
3251
+ },
3252
+ {
3253
+ "epoch": 48.596564725596984,
3254
+ "grad_norm": 4.660221099853516,
3255
+ "learning_rate": 1.4233347297863427e-06,
3256
+ "loss": 5.7742,
3257
+ "step": 232000
3258
+ },
3259
+ {
3260
+ "epoch": 48.701298701298704,
3261
+ "grad_norm": 6.032063961029053,
3262
+ "learning_rate": 1.3188102220360285e-06,
3263
+ "loss": 5.7866,
3264
+ "step": 232500
3265
+ },
3266
+ {
3267
+ "epoch": 48.80603267700042,
3268
+ "grad_norm": 6.296925067901611,
3269
+ "learning_rate": 1.2142857142857144e-06,
3270
+ "loss": 5.7922,
3271
+ "step": 233000
3272
+ },
3273
+ {
3274
+ "epoch": 48.91076665270214,
3275
+ "grad_norm": 5.2991437911987305,
3276
+ "learning_rate": 1.1095517385839968e-06,
3277
+ "loss": 5.7873,
3278
+ "step": 233500
3279
+ },
3280
+ {
3281
+ "epoch": 49.015500628403856,
3282
+ "grad_norm": 6.130777835845947,
3283
+ "learning_rate": 1.0048177628822792e-06,
3284
+ "loss": 5.8074,
3285
+ "step": 234000
3286
+ },
3287
+ {
3288
+ "epoch": 49.12023460410557,
3289
+ "grad_norm": 6.305094242095947,
3290
+ "learning_rate": 9.000837871805614e-07,
3291
+ "loss": 5.7847,
3292
+ "step": 234500
3293
+ },
3294
+ {
3295
+ "epoch": 49.22496857980729,
3296
+ "grad_norm": 6.156949996948242,
3297
+ "learning_rate": 7.953498114788438e-07,
3298
+ "loss": 5.7741,
3299
+ "step": 235000
3300
+ },
3301
+ {
3302
+ "epoch": 49.32970255550901,
3303
+ "grad_norm": 6.475966930389404,
3304
+ "learning_rate": 6.908253037285296e-07,
3305
+ "loss": 5.8083,
3306
+ "step": 235500
3307
+ },
3308
+ {
3309
+ "epoch": 49.43443653121072,
3310
+ "grad_norm": 5.687895774841309,
3311
+ "learning_rate": 5.86091328026812e-07,
3312
+ "loss": 5.7976,
3313
+ "step": 236000
3314
+ },
3315
+ {
3316
+ "epoch": 49.53917050691244,
3317
+ "grad_norm": 5.615401744842529,
3318
+ "learning_rate": 4.813573523250943e-07,
3319
+ "loss": 5.7907,
3320
+ "step": 236500
3321
+ },
3322
+ {
3323
+ "epoch": 49.64390448261416,
3324
+ "grad_norm": 6.177160263061523,
3325
+ "learning_rate": 3.7662337662337666e-07,
3326
+ "loss": 5.792,
3327
+ "step": 237000
3328
+ },
3329
+ {
3330
+ "epoch": 49.74863845831588,
3331
+ "grad_norm": 5.604287624359131,
3332
+ "learning_rate": 2.71889400921659e-07,
3333
+ "loss": 5.7837,
3334
+ "step": 237500
3335
+ },
3336
+ {
3337
+ "epoch": 49.853372434017594,
3338
+ "grad_norm": 4.826539039611816,
3339
+ "learning_rate": 1.673648931713448e-07,
3340
+ "loss": 5.7808,
3341
+ "step": 238000
3342
+ },
3343
+ {
3344
+ "epoch": 49.95810640971931,
3345
+ "grad_norm": 5.6649250984191895,
3346
+ "learning_rate": 6.263091746962715e-08,
3347
+ "loss": 5.7888,
3348
+ "step": 238500
3349
+ },
3350
+ {
3351
+ "epoch": 50.0,
3352
+ "step": 238700,
3353
+ "total_flos": 5161725447936000.0,
3354
+ "train_loss": 6.140905278960581,
3355
+ "train_runtime": 7883.0646,
3356
+ "train_samples_per_second": 484.444,
3357
+ "train_steps_per_second": 30.28
3358
+ }
3359
+ ],
3360
+ "logging_steps": 500,
3361
+ "max_steps": 238700,
3362
+ "num_input_tokens_seen": 0,
3363
+ "num_train_epochs": 50,
3364
+ "save_steps": 5000,
3365
+ "stateful_callbacks": {
3366
+ "TrainerControl": {
3367
+ "args": {
3368
+ "should_epoch_stop": false,
3369
+ "should_evaluate": false,
3370
+ "should_log": false,
3371
+ "should_save": true,
3372
+ "should_training_stop": true
3373
+ },
3374
+ "attributes": {}
3375
+ }
3376
+ },
3377
+ "total_flos": 5161725447936000.0,
3378
+ "train_batch_size": 16,
3379
+ "trial_name": null,
3380
+ "trial_params": null
3381
+ }