Training in progress, step 239, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-Math-7B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-Math-7B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "embed_tokens",
+    "lm_head"
+  ],
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "down_proj",
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db845de3d17c9726c0430f3055f4e76a262fe10c865c4ba50ee36716e68a8c77
+size 2503003904

last-checkpoint/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a642510089069eb8b814518276f9ee0518a178090ca6e10812c0a021bb794ff7
+size 5006244836

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cbb69e640b033706cbbbfb4d8b1ef3c3d331a2c28c0fa976fc70e907da2ceda
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6700a68575e0f304dc8dd30cc031ed878528ade38577f30c04efe60a01876a21
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1706 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.5278851463279955,
+  "eval_steps": 500,
+  "global_step": 239,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0022087244616234127,
+      "grad_norm": 5.535903453826904,
+      "learning_rate": 1.0000000000000002e-06,
+      "loss": 6.0457,
+      "step": 1
+    },
+    {
+      "epoch": 0.004417448923246825,
+      "grad_norm": 6.602200984954834,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 8.2904,
+      "step": 2
+    },
+    {
+      "epoch": 0.006626173384870237,
+      "grad_norm": 6.935258865356445,
+      "learning_rate": 3e-06,
+      "loss": 8.4242,
+      "step": 3
+    },
+    {
+      "epoch": 0.00883489784649365,
+      "grad_norm": 6.811558723449707,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 8.2953,
+      "step": 4
+    },
+    {
+      "epoch": 0.011043622308117063,
+      "grad_norm": 6.635751247406006,
+      "learning_rate": 5e-06,
+      "loss": 8.186,
+      "step": 5
+    },
+    {
+      "epoch": 0.013252346769740474,
+      "grad_norm": 7.359472751617432,
+      "learning_rate": 6e-06,
+      "loss": 8.2423,
+      "step": 6
+    },
+    {
+      "epoch": 0.015461071231363888,
+      "grad_norm": 7.092770099639893,
+      "learning_rate": 7.000000000000001e-06,
+      "loss": 8.6333,
+      "step": 7
+    },
+    {
+      "epoch": 0.0176697956929873,
+      "grad_norm": 7.17343282699585,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 8.2698,
+      "step": 8
+    },
+    {
+      "epoch": 0.019878520154610713,
+      "grad_norm": 7.231985092163086,
+      "learning_rate": 9e-06,
+      "loss": 8.0394,
+      "step": 9
+    },
+    {
+      "epoch": 0.022087244616234125,
+      "grad_norm": 7.38832426071167,
+      "learning_rate": 1e-05,
+      "loss": 8.3605,
+      "step": 10
+    },
+    {
+      "epoch": 0.024295969077857537,
+      "grad_norm": 7.6347551345825195,
+      "learning_rate": 1.1000000000000001e-05,
+      "loss": 8.4294,
+      "step": 11
+    },
+    {
+      "epoch": 0.02650469353948095,
+      "grad_norm": 7.681893825531006,
+      "learning_rate": 1.2e-05,
+      "loss": 8.2903,
+      "step": 12
+    },
+    {
+      "epoch": 0.02871341800110436,
+      "grad_norm": 7.411649227142334,
+      "learning_rate": 1.3000000000000001e-05,
+      "loss": 8.5525,
+      "step": 13
+    },
+    {
+      "epoch": 0.030922142462727776,
+      "grad_norm": 7.768363952636719,
+      "learning_rate": 1.4000000000000001e-05,
+      "loss": 8.7325,
+      "step": 14
+    },
+    {
+      "epoch": 0.03313086692435119,
+      "grad_norm": 8.075197219848633,
+      "learning_rate": 1.5e-05,
+      "loss": 8.48,
+      "step": 15
+    },
+    {
+      "epoch": 0.0353395913859746,
+      "grad_norm": 7.757429122924805,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 8.3817,
+      "step": 16
+    },
+    {
+      "epoch": 0.03754831584759801,
+      "grad_norm": 8.141379356384277,
+      "learning_rate": 1.7000000000000003e-05,
+      "loss": 8.5635,
+      "step": 17
+    },
+    {
+      "epoch": 0.03975704030922143,
+      "grad_norm": 8.649480819702148,
+      "learning_rate": 1.8e-05,
+      "loss": 8.6707,
+      "step": 18
+    },
+    {
+      "epoch": 0.041965764770844835,
+      "grad_norm": 8.826072692871094,
+      "learning_rate": 1.9e-05,
+      "loss": 8.8567,
+      "step": 19
+    },
+    {
+      "epoch": 0.04417448923246825,
+      "grad_norm": 8.904451370239258,
+      "learning_rate": 2e-05,
+      "loss": 8.5523,
+      "step": 20
+    },
+    {
+      "epoch": 0.04638321369409166,
+      "grad_norm": 9.640382766723633,
+      "learning_rate": 2.1e-05,
+      "loss": 8.3306,
+      "step": 21
+    },
+    {
+      "epoch": 0.048591938155715074,
+      "grad_norm": 9.720635414123535,
+      "learning_rate": 2.2000000000000003e-05,
+      "loss": 8.4766,
+      "step": 22
+    },
+    {
+      "epoch": 0.05080066261733849,
+      "grad_norm": 10.471412658691406,
+      "learning_rate": 2.3000000000000003e-05,
+      "loss": 8.4523,
+      "step": 23
+    },
+    {
+      "epoch": 0.0530093870789619,
+      "grad_norm": 10.258910179138184,
+      "learning_rate": 2.4e-05,
+      "loss": 8.5165,
+      "step": 24
+    },
+    {
+      "epoch": 0.05521811154058531,
+      "grad_norm": 10.779662132263184,
+      "learning_rate": 2.5e-05,
+      "loss": 8.4274,
+      "step": 25
+    },
+    {
+      "epoch": 0.05742683600220872,
+      "grad_norm": 10.864973068237305,
+      "learning_rate": 2.6000000000000002e-05,
+      "loss": 8.1579,
+      "step": 26
+    },
+    {
+      "epoch": 0.05963556046383214,
+      "grad_norm": 11.23272705078125,
+      "learning_rate": 2.7000000000000002e-05,
+      "loss": 8.3138,
+      "step": 27
+    },
+    {
+      "epoch": 0.06184428492545555,
+      "grad_norm": 11.67050838470459,
+      "learning_rate": 2.8000000000000003e-05,
+      "loss": 8.3039,
+      "step": 28
+    },
+    {
+      "epoch": 0.06405300938707896,
+      "grad_norm": 11.947738647460938,
+      "learning_rate": 2.9e-05,
+      "loss": 8.0931,
+      "step": 29
+    },
+    {
+      "epoch": 0.06626173384870238,
+      "grad_norm": 12.407663345336914,
+      "learning_rate": 3e-05,
+      "loss": 7.9797,
+      "step": 30
+    },
+    {
+      "epoch": 0.06847045831032579,
+      "grad_norm": 12.427129745483398,
+      "learning_rate": 3.1e-05,
+      "loss": 8.3415,
+      "step": 31
+    },
+    {
+      "epoch": 0.0706791827719492,
+      "grad_norm": 13.105308532714844,
+      "learning_rate": 3.2000000000000005e-05,
+      "loss": 7.8873,
+      "step": 32
+    },
+    {
+      "epoch": 0.07288790723357261,
+      "grad_norm": 12.776379585266113,
+      "learning_rate": 3.3e-05,
+      "loss": 7.837,
+      "step": 33
+    },
+    {
+      "epoch": 0.07509663169519602,
+      "grad_norm": 13.840560913085938,
+      "learning_rate": 3.4000000000000007e-05,
+      "loss": 7.5128,
+      "step": 34
+    },
+    {
+      "epoch": 0.07730535615681944,
+      "grad_norm": 12.904688835144043,
+      "learning_rate": 3.5e-05,
+      "loss": 7.5659,
+      "step": 35
+    },
+    {
+      "epoch": 0.07951408061844285,
+      "grad_norm": 12.567195892333984,
+      "learning_rate": 3.6e-05,
+      "loss": 7.7097,
+      "step": 36
+    },
+    {
+      "epoch": 0.08172280508006627,
+      "grad_norm": 11.897880554199219,
+      "learning_rate": 3.7e-05,
+      "loss": 7.2553,
+      "step": 37
+    },
+    {
+      "epoch": 0.08393152954168967,
+      "grad_norm": 13.644888877868652,
+      "learning_rate": 3.8e-05,
+      "loss": 7.6672,
+      "step": 38
+    },
+    {
+      "epoch": 0.08614025400331309,
+      "grad_norm": 12.215312004089355,
+      "learning_rate": 3.9000000000000006e-05,
+      "loss": 7.321,
+      "step": 39
+    },
+    {
+      "epoch": 0.0883489784649365,
+      "grad_norm": 11.68522834777832,
+      "learning_rate": 4e-05,
+      "loss": 7.67,
+      "step": 40
+    },
+    {
+      "epoch": 0.09055770292655992,
+      "grad_norm": 11.208578109741211,
+      "learning_rate": 4.1e-05,
+      "loss": 7.6815,
+      "step": 41
+    },
+    {
+      "epoch": 0.09276642738818332,
+      "grad_norm": 11.380876541137695,
+      "learning_rate": 4.2e-05,
+      "loss": 7.7252,
+      "step": 42
+    },
+    {
+      "epoch": 0.09497515184980673,
+      "grad_norm": 10.951228141784668,
+      "learning_rate": 4.3e-05,
+      "loss": 7.2269,
+      "step": 43
+    },
+    {
+      "epoch": 0.09718387631143015,
+      "grad_norm": 10.96963882446289,
+      "learning_rate": 4.4000000000000006e-05,
+      "loss": 6.5408,
+      "step": 44
+    },
+    {
+      "epoch": 0.09939260077305356,
+      "grad_norm": 11.59006404876709,
+      "learning_rate": 4.5e-05,
+      "loss": 6.9152,
+      "step": 45
+    },
+    {
+      "epoch": 0.10160132523467698,
+      "grad_norm": 11.817084312438965,
+      "learning_rate": 4.600000000000001e-05,
+      "loss": 6.6265,
+      "step": 46
+    },
+    {
+      "epoch": 0.10381004969630038,
+      "grad_norm": 12.219128608703613,
+      "learning_rate": 4.7e-05,
+      "loss": 6.6341,
+      "step": 47
+    },
+    {
+      "epoch": 0.1060187741579238,
+      "grad_norm": 12.786672592163086,
+      "learning_rate": 4.8e-05,
+      "loss": 6.7037,
+      "step": 48
+    },
+    {
+      "epoch": 0.10822749861954721,
+      "grad_norm": 12.815557479858398,
+      "learning_rate": 4.9e-05,
+      "loss": 6.7343,
+      "step": 49
+    },
+    {
+      "epoch": 0.11043622308117063,
+      "grad_norm": 12.737327575683594,
+      "learning_rate": 5e-05,
+      "loss": 7.3387,
+      "step": 50
+    },
+    {
+      "epoch": 0.11264494754279404,
+      "grad_norm": 6.819960594177246,
+      "learning_rate": 5.1000000000000006e-05,
+      "loss": 4.7037,
+      "step": 51
+    },
+    {
+      "epoch": 0.11485367200441744,
+      "grad_norm": 12.15744686126709,
+      "learning_rate": 5.2000000000000004e-05,
+      "loss": 6.148,
+      "step": 52
+    },
+    {
+      "epoch": 0.11706239646604086,
+      "grad_norm": 11.386445045471191,
+      "learning_rate": 5.300000000000001e-05,
+      "loss": 6.0605,
+      "step": 53
+    },
+    {
+      "epoch": 0.11927112092766427,
+      "grad_norm": 11.03603458404541,
+      "learning_rate": 5.4000000000000005e-05,
+      "loss": 6.1902,
+      "step": 54
+    },
+    {
+      "epoch": 0.12147984538928769,
+      "grad_norm": 10.725998878479004,
+      "learning_rate": 5.500000000000001e-05,
+      "loss": 5.8361,
+      "step": 55
+    },
+    {
+      "epoch": 0.1236885698509111,
+      "grad_norm": 8.622870445251465,
+      "learning_rate": 5.6000000000000006e-05,
+      "loss": 5.865,
+      "step": 56
+    },
+    {
+      "epoch": 0.12589729431253452,
+      "grad_norm": 7.68513298034668,
+      "learning_rate": 5.6999999999999996e-05,
+      "loss": 6.1753,
+      "step": 57
+    },
+    {
+      "epoch": 0.12810601877415792,
+      "grad_norm": 7.354230880737305,
+      "learning_rate": 5.8e-05,
+      "loss": 5.8207,
+      "step": 58
+    },
+    {
+      "epoch": 0.13031474323578135,
+      "grad_norm": 6.177774906158447,
+      "learning_rate": 5.9e-05,
+      "loss": 5.7267,
+      "step": 59
+    },
+    {
+      "epoch": 0.13252346769740475,
+      "grad_norm": 6.288235187530518,
+      "learning_rate": 6e-05,
+      "loss": 5.7419,
+      "step": 60
+    },
+    {
+      "epoch": 0.13473219215902815,
+      "grad_norm": 6.973496437072754,
+      "learning_rate": 6.1e-05,
+      "loss": 5.8684,
+      "step": 61
+    },
+    {
+      "epoch": 0.13694091662065158,
+      "grad_norm": 6.646786689758301,
+      "learning_rate": 6.2e-05,
+      "loss": 5.6718,
+      "step": 62
+    },
+    {
+      "epoch": 0.13914964108227498,
+      "grad_norm": 7.017954349517822,
+      "learning_rate": 6.3e-05,
+      "loss": 5.6098,
+      "step": 63
+    },
+    {
+      "epoch": 0.1413583655438984,
+      "grad_norm": 7.299660682678223,
+      "learning_rate": 6.400000000000001e-05,
+      "loss": 5.6966,
+      "step": 64
+    },
+    {
+      "epoch": 0.1435670900055218,
+      "grad_norm": 8.129739761352539,
+      "learning_rate": 6.500000000000001e-05,
+      "loss": 5.8456,
+      "step": 65
+    },
+    {
+      "epoch": 0.14577581446714521,
+      "grad_norm": 8.626437187194824,
+      "learning_rate": 6.6e-05,
+      "loss": 5.9174,
+      "step": 66
+    },
+    {
+      "epoch": 0.14798453892876864,
+      "grad_norm": 9.129312515258789,
+      "learning_rate": 6.7e-05,
+      "loss": 5.7032,
+      "step": 67
+    },
+    {
+      "epoch": 0.15019326339039205,
+      "grad_norm": 8.709549903869629,
+      "learning_rate": 6.800000000000001e-05,
+      "loss": 5.8617,
+      "step": 68
+    },
+    {
+      "epoch": 0.15240198785201547,
+      "grad_norm": 9.02973747253418,
+      "learning_rate": 6.9e-05,
+      "loss": 5.8936,
+      "step": 69
+    },
+    {
+      "epoch": 0.15461071231363888,
+      "grad_norm": 8.83764934539795,
+      "learning_rate": 7e-05,
+      "loss": 5.7209,
+      "step": 70
+    },
+    {
+      "epoch": 0.15681943677526228,
+      "grad_norm": 9.076760292053223,
+      "learning_rate": 7.1e-05,
+      "loss": 5.7627,
+      "step": 71
+    },
+    {
+      "epoch": 0.1590281612368857,
+      "grad_norm": 9.358622550964355,
+      "learning_rate": 7.2e-05,
+      "loss": 5.9458,
+      "step": 72
+    },
+    {
+      "epoch": 0.1612368856985091,
+      "grad_norm": 8.883482933044434,
+      "learning_rate": 7.3e-05,
+      "loss": 5.6288,
+      "step": 73
+    },
+    {
+      "epoch": 0.16344561016013254,
+      "grad_norm": 9.588410377502441,
+      "learning_rate": 7.4e-05,
+      "loss": 5.5858,
+      "step": 74
+    },
+    {
+      "epoch": 0.16565433462175594,
+      "grad_norm": 10.137778282165527,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 5.5368,
+      "step": 75
+    },
+    {
+      "epoch": 0.16786305908337934,
+      "grad_norm": 10.220065116882324,
+      "learning_rate": 7.6e-05,
+      "loss": 5.7181,
+      "step": 76
+    },
+    {
+      "epoch": 0.17007178354500277,
+      "grad_norm": 9.702897071838379,
+      "learning_rate": 7.7e-05,
+      "loss": 5.6469,
+      "step": 77
+    },
+    {
+      "epoch": 0.17228050800662617,
+      "grad_norm": 9.3972806930542,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 5.4397,
+      "step": 78
+    },
+    {
+      "epoch": 0.17448923246824957,
+      "grad_norm": 10.074295043945312,
+      "learning_rate": 7.900000000000001e-05,
+      "loss": 5.397,
+      "step": 79
+    },
+    {
+      "epoch": 0.176697956929873,
+      "grad_norm": 10.400197982788086,
+      "learning_rate": 8e-05,
+      "loss": 5.5235,
+      "step": 80
+    },
+    {
+      "epoch": 0.1789066813914964,
+      "grad_norm": 10.35517406463623,
+      "learning_rate": 8.1e-05,
+      "loss": 5.6897,
+      "step": 81
+    },
+    {
+      "epoch": 0.18111540585311983,
+      "grad_norm": 11.12540054321289,
+      "learning_rate": 8.2e-05,
+      "loss": 5.5081,
+      "step": 82
+    },
+    {
+      "epoch": 0.18332413031474323,
+      "grad_norm": 11.132303237915039,
+      "learning_rate": 8.3e-05,
+      "loss": 5.8882,
+      "step": 83
+    },
+    {
+      "epoch": 0.18553285477636663,
+      "grad_norm": 10.35818099975586,
+      "learning_rate": 8.4e-05,
+      "loss": 5.1969,
+      "step": 84
+    },
+    {
+      "epoch": 0.18774157923799006,
+      "grad_norm": 11.621879577636719,
+      "learning_rate": 8.5e-05,
+      "loss": 5.6639,
+      "step": 85
+    },
+    {
+      "epoch": 0.18995030369961347,
+      "grad_norm": 11.88983154296875,
+      "learning_rate": 8.6e-05,
+      "loss": 5.7786,
+      "step": 86
+    },
+    {
+      "epoch": 0.1921590281612369,
+      "grad_norm": 11.17719841003418,
+      "learning_rate": 8.7e-05,
+      "loss": 5.5071,
+      "step": 87
+    },
+    {
+      "epoch": 0.1943677526228603,
+      "grad_norm": 11.159234046936035,
+      "learning_rate": 8.800000000000001e-05,
+      "loss": 5.4638,
+      "step": 88
+    },
+    {
+      "epoch": 0.1965764770844837,
+      "grad_norm": 10.719842910766602,
+      "learning_rate": 8.900000000000001e-05,
+      "loss": 5.3603,
+      "step": 89
+    },
+    {
+      "epoch": 0.19878520154610713,
+      "grad_norm": 10.188541412353516,
+      "learning_rate": 9e-05,
+      "loss": 5.7355,
+      "step": 90
+    },
+    {
+      "epoch": 0.20099392600773053,
+      "grad_norm": 10.070486068725586,
+      "learning_rate": 9.1e-05,
+      "loss": 5.1332,
+      "step": 91
+    },
+    {
+      "epoch": 0.20320265046935396,
+      "grad_norm": 10.740564346313477,
+      "learning_rate": 9.200000000000001e-05,
+      "loss": 5.6564,
+      "step": 92
+    },
+    {
+      "epoch": 0.20541137493097736,
+      "grad_norm": 11.773510932922363,
+      "learning_rate": 9.300000000000001e-05,
+      "loss": 5.3035,
+      "step": 93
+    },
+    {
+      "epoch": 0.20762009939260076,
+      "grad_norm": 10.705671310424805,
+      "learning_rate": 9.4e-05,
+      "loss": 5.4218,
+      "step": 94
+    },
+    {
+      "epoch": 0.2098288238542242,
+      "grad_norm": 10.64268684387207,
+      "learning_rate": 9.5e-05,
+      "loss": 5.2128,
+      "step": 95
+    },
+    {
+      "epoch": 0.2120375483158476,
+      "grad_norm": 11.824028015136719,
+      "learning_rate": 9.6e-05,
+      "loss": 5.5192,
+      "step": 96
+    },
+    {
+      "epoch": 0.21424627277747102,
+      "grad_norm": 10.851103782653809,
+      "learning_rate": 9.7e-05,
+      "loss": 5.6141,
+      "step": 97
+    },
+    {
+      "epoch": 0.21645499723909442,
+      "grad_norm": 11.880955696105957,
+      "learning_rate": 9.8e-05,
+      "loss": 5.7783,
+      "step": 98
+    },
+    {
+      "epoch": 0.21866372170071782,
+      "grad_norm": 11.984313011169434,
+      "learning_rate": 9.900000000000001e-05,
+      "loss": 5.1242,
+      "step": 99
+    },
+    {
+      "epoch": 0.22087244616234125,
+      "grad_norm": 12.726852416992188,
+      "learning_rate": 0.0001,
+      "loss": 5.1561,
+      "step": 100
+    },
+    {
+      "epoch": 0.22308117062396465,
+      "grad_norm": 7.531022548675537,
+      "learning_rate": 9.999800863190707e-05,
+      "loss": 4.6879,
+      "step": 101
+    },
+    {
+      "epoch": 0.22528989508558808,
+      "grad_norm": 9.076756477355957,
+      "learning_rate": 9.999203468625017e-05,
+      "loss": 5.4269,
+      "step": 102
+    },
+    {
+      "epoch": 0.22749861954721148,
+      "grad_norm": 7.9871954917907715,
+      "learning_rate": 9.998207863888228e-05,
+      "loss": 5.3304,
+      "step": 103
+    },
+    {
+      "epoch": 0.22970734400883489,
+      "grad_norm": 7.338669300079346,
+      "learning_rate": 9.99681412828496e-05,
+      "loss": 5.1078,
+      "step": 104
+    },
+    {
+      "epoch": 0.23191606847045831,
+      "grad_norm": 5.404487609863281,
+      "learning_rate": 9.995022372832836e-05,
+      "loss": 4.879,
+      "step": 105
+    },
+    {
+      "epoch": 0.23412479293208172,
+      "grad_norm": 5.911509037017822,
+      "learning_rate": 9.992832740253645e-05,
+      "loss": 4.5742,
+      "step": 106
+    },
+    {
+      "epoch": 0.23633351739370514,
+      "grad_norm": 5.688644886016846,
+      "learning_rate": 9.990245404961963e-05,
+      "loss": 4.9074,
+      "step": 107
+    },
+    {
+      "epoch": 0.23854224185532855,
+      "grad_norm": 5.671441078186035,
+      "learning_rate": 9.987260573051269e-05,
+      "loss": 4.8472,
+      "step": 108
+    },
+    {
+      "epoch": 0.24075096631695195,
+      "grad_norm": 6.069337368011475,
+      "learning_rate": 9.983878482277522e-05,
+      "loss": 4.866,
+      "step": 109
+    },
+    {
+      "epoch": 0.24295969077857538,
+      "grad_norm": 6.0655694007873535,
+      "learning_rate": 9.98009940204023e-05,
+      "loss": 4.9271,
+      "step": 110
+    },
+    {
+      "epoch": 0.24516841524019878,
+      "grad_norm": 6.612443447113037,
+      "learning_rate": 9.975923633360985e-05,
+      "loss": 5.0706,
+      "step": 111
+    },
+    {
+      "epoch": 0.2473771397018222,
+      "grad_norm": 6.772550582885742,
+      "learning_rate": 9.971351508859488e-05,
+      "loss": 5.1031,
+      "step": 112
+    },
+    {
+      "epoch": 0.2495858641634456,
+      "grad_norm": 6.7489800453186035,
+      "learning_rate": 9.96638339272705e-05,
+      "loss": 5.3257,
+      "step": 113
+    },
+    {
+      "epoch": 0.25179458862506904,
+      "grad_norm": 6.750669002532959,
+      "learning_rate": 9.961019680697593e-05,
+      "loss": 5.0159,
+      "step": 114
+    },
+    {
+      "epoch": 0.25400331308669244,
+      "grad_norm": 7.035401821136475,
+      "learning_rate": 9.955260800016113e-05,
+      "loss": 4.9488,
+      "step": 115
+    },
+    {
+      "epoch": 0.25621203754831584,
+      "grad_norm": 7.1767473220825195,
+      "learning_rate": 9.949107209404665e-05,
+      "loss": 4.7798,
+      "step": 116
+    },
+    {
+      "epoch": 0.25842076200993924,
+      "grad_norm": 7.472834587097168,
+      "learning_rate": 9.942559399025803e-05,
+      "loss": 4.9628,
+      "step": 117
+    },
+    {
+      "epoch": 0.2606294864715627,
+      "grad_norm": 7.51600456237793,
+      "learning_rate": 9.935617890443557e-05,
+      "loss": 5.4082,
+      "step": 118
+    },
+    {
+      "epoch": 0.2628382109331861,
+      "grad_norm": 7.743865489959717,
+      "learning_rate": 9.928283236581873e-05,
+      "loss": 5.2246,
+      "step": 119
+    },
+    {
+      "epoch": 0.2650469353948095,
+      "grad_norm": 7.967807292938232,
+      "learning_rate": 9.92055602168058e-05,
+      "loss": 5.0226,
+      "step": 120
+    },
+    {
+      "epoch": 0.2672556598564329,
+      "grad_norm": 8.627487182617188,
+      "learning_rate": 9.912436861248847e-05,
+      "loss": 5.1528,
+      "step": 121
+    },
+    {
+      "epoch": 0.2694643843180563,
+      "grad_norm": 8.125624656677246,
+      "learning_rate": 9.903926402016153e-05,
+      "loss": 5.3868,
+      "step": 122
+    },
+    {
+      "epoch": 0.27167310877967976,
+      "grad_norm": 8.606098175048828,
+      "learning_rate": 9.895025321880776e-05,
+      "loss": 5.1796,
+      "step": 123
+    },
+    {
+      "epoch": 0.27388183324130316,
+      "grad_norm": 8.127717971801758,
+      "learning_rate": 9.885734329855798e-05,
+      "loss": 4.8976,
+      "step": 124
+    },
+    {
+      "epoch": 0.27609055770292656,
+      "grad_norm": 8.626016616821289,
+      "learning_rate": 9.87605416601262e-05,
+      "loss": 4.9745,
+      "step": 125
+    },
+    {
+      "epoch": 0.27829928216454997,
+      "grad_norm": 8.542702674865723,
+      "learning_rate": 9.865985601422018e-05,
+      "loss": 4.7371,
+      "step": 126
+    },
+    {
+      "epoch": 0.28050800662617337,
+      "grad_norm": 8.690094947814941,
+      "learning_rate": 9.855529438092724e-05,
+      "loss": 5.1957,
+      "step": 127
+    },
+    {
+      "epoch": 0.2827167310877968,
+      "grad_norm": 8.771077156066895,
+      "learning_rate": 9.844686508907537e-05,
+      "loss": 4.7499,
+      "step": 128
+    },
+    {
+      "epoch": 0.2849254555494202,
+      "grad_norm": 8.793961524963379,
+      "learning_rate": 9.833457677556987e-05,
+      "loss": 5.3736,
+      "step": 129
+    },
+    {
+      "epoch": 0.2871341800110436,
+      "grad_norm": 9.016656875610352,
+      "learning_rate": 9.821843838470534e-05,
+      "loss": 5.0202,
+      "step": 130
+    },
+    {
+      "epoch": 0.28934290447266703,
+      "grad_norm": 8.994014739990234,
+      "learning_rate": 9.809845916745321e-05,
+      "loss": 4.9291,
+      "step": 131
+    },
+    {
+      "epoch": 0.29155162893429043,
+      "grad_norm": 8.716841697692871,
+      "learning_rate": 9.797464868072488e-05,
+      "loss": 4.9976,
+      "step": 132
+    },
+    {
+      "epoch": 0.2937603533959139,
+      "grad_norm": 9.860023498535156,
+      "learning_rate": 9.784701678661045e-05,
+      "loss": 4.553,
+      "step": 133
+    },
+    {
+      "epoch": 0.2959690778575373,
+      "grad_norm": 9.492415428161621,
+      "learning_rate": 9.77155736515932e-05,
+      "loss": 5.1861,
+      "step": 134
+    },
+    {
+      "epoch": 0.2981778023191607,
+      "grad_norm": 9.687310218811035,
+      "learning_rate": 9.758032974573972e-05,
+      "loss": 5.3625,
+      "step": 135
+    },
+    {
+      "epoch": 0.3003865267807841,
+      "grad_norm": 9.293684959411621,
+      "learning_rate": 9.744129584186598e-05,
+      "loss": 5.2148,
+      "step": 136
+    },
+    {
+      "epoch": 0.3025952512424075,
+      "grad_norm": 9.503033638000488,
+      "learning_rate": 9.729848301467919e-05,
+      "loss": 5.5468,
+      "step": 137
+    },
+    {
+      "epoch": 0.30480397570403095,
+      "grad_norm": 9.93913459777832,
+      "learning_rate": 9.715190263989561e-05,
+      "loss": 5.2361,
+      "step": 138
+    },
+    {
+      "epoch": 0.30701270016565435,
+      "grad_norm": 8.820573806762695,
+      "learning_rate": 9.700156639333451e-05,
+      "loss": 4.6691,
+      "step": 139
+    },
+    {
+      "epoch": 0.30922142462727775,
+      "grad_norm": 9.540395736694336,
+      "learning_rate": 9.68474862499881e-05,
+      "loss": 4.5415,
+      "step": 140
+    },
+    {
+      "epoch": 0.31143014908890115,
+      "grad_norm": 9.923493385314941,
+      "learning_rate": 9.668967448306758e-05,
+      "loss": 4.8208,
+      "step": 141
+    },
+    {
+      "epoch": 0.31363887355052456,
+      "grad_norm": 10.182249069213867,
+      "learning_rate": 9.652814366302568e-05,
+      "loss": 5.0238,
+      "step": 142
+    },
+    {
+      "epoch": 0.315847598012148,
+      "grad_norm": 9.629402160644531,
+      "learning_rate": 9.636290665655524e-05,
+      "loss": 5.072,
+      "step": 143
+    },
+    {
+      "epoch": 0.3180563224737714,
+      "grad_norm": 10.206043243408203,
+      "learning_rate": 9.619397662556435e-05,
+      "loss": 5.0429,
+      "step": 144
+    },
+    {
+      "epoch": 0.3202650469353948,
+      "grad_norm": 10.271145820617676,
+      "learning_rate": 9.602136702612794e-05,
+      "loss": 5.6498,
+      "step": 145
+    },
+    {
+      "epoch": 0.3224737713970182,
+      "grad_norm": 9.493379592895508,
+      "learning_rate": 9.584509160741599e-05,
+      "loss": 4.8689,
+      "step": 146
+    },
+    {
+      "epoch": 0.3246824958586416,
+      "grad_norm": 10.454998016357422,
+      "learning_rate": 9.566516441059827e-05,
+      "loss": 5.7323,
+      "step": 147
+    },
+    {
+      "epoch": 0.3268912203202651,
+      "grad_norm": 11.290060997009277,
+      "learning_rate": 9.548159976772592e-05,
+      "loss": 5.1861,
+      "step": 148
+    },
+    {
+      "epoch": 0.3290999447818885,
+      "grad_norm": 10.804366111755371,
+      "learning_rate": 9.529441230058987e-05,
+      "loss": 5.0733,
+      "step": 149
+    },
+    {
+      "epoch": 0.3313086692435119,
+      "grad_norm": 11.59500503540039,
+      "learning_rate": 9.510361691955608e-05,
+      "loss": 5.185,
+      "step": 150
+    },
+    {
+      "epoch": 0.3335173937051353,
+      "grad_norm": 5.955521106719971,
+      "learning_rate": 9.490922882237791e-05,
+      "loss": 3.7304,
+      "step": 151
+    },
+    {
+      "epoch": 0.3357261181667587,
+      "grad_norm": 6.95432710647583,
+      "learning_rate": 9.471126349298556e-05,
+      "loss": 5.0306,
+      "step": 152
+    },
+    {
+      "epoch": 0.3379348426283821,
+      "grad_norm": 7.6052937507629395,
+      "learning_rate": 9.450973670025264e-05,
+      "loss": 4.8507,
+      "step": 153
+    },
+    {
+      "epoch": 0.34014356709000554,
+      "grad_norm": 7.37522029876709,
+      "learning_rate": 9.430466449674014e-05,
+      "loss": 4.9886,
+      "step": 154
+    },
+    {
+      "epoch": 0.34235229155162894,
+      "grad_norm": 6.831510543823242,
+      "learning_rate": 9.409606321741775e-05,
+      "loss": 4.9264,
+      "step": 155
+    },
+    {
+      "epoch": 0.34456101601325234,
+      "grad_norm": 6.073673248291016,
+      "learning_rate": 9.388394947836279e-05,
+      "loss": 4.7457,
+      "step": 156
+    },
+    {
+      "epoch": 0.34676974047487574,
+      "grad_norm": 5.337328910827637,
+      "learning_rate": 9.366834017543651e-05,
+      "loss": 4.7442,
+      "step": 157
+    },
+    {
+      "epoch": 0.34897846493649914,
+      "grad_norm": 5.276707649230957,
+      "learning_rate": 9.344925248293837e-05,
+      "loss": 4.9261,
+      "step": 158
+    },
+    {
+      "epoch": 0.3511871893981226,
+      "grad_norm": 5.147680759429932,
+      "learning_rate": 9.322670385223797e-05,
+      "loss": 4.4692,
+      "step": 159
+    },
+    {
+      "epoch": 0.353395913859746,
+      "grad_norm": 5.277919292449951,
+      "learning_rate": 9.300071201038503e-05,
+      "loss": 4.8624,
+      "step": 160
+    },
+    {
+      "epoch": 0.3556046383213694,
+      "grad_norm": 5.418294906616211,
+      "learning_rate": 9.277129495869726e-05,
+      "loss": 4.6649,
+      "step": 161
+    },
+    {
+      "epoch": 0.3578133627829928,
+      "grad_norm": 5.715703964233398,
+      "learning_rate": 9.253847097132655e-05,
+      "loss": 5.2021,
+      "step": 162
+    },
+    {
+      "epoch": 0.3600220872446162,
+      "grad_norm": 5.969396591186523,
+      "learning_rate": 9.230225859380326e-05,
+      "loss": 5.0423,
+      "step": 163
+    },
+    {
+      "epoch": 0.36223081170623966,
+      "grad_norm": 5.960832118988037,
+      "learning_rate": 9.206267664155907e-05,
+      "loss": 4.8568,
+      "step": 164
+    },
+    {
+      "epoch": 0.36443953616786307,
+      "grad_norm": 6.366528034210205,
+      "learning_rate": 9.181974419842818e-05,
+      "loss": 4.7735,
+      "step": 165
+    },
+    {
+      "epoch": 0.36664826062948647,
+      "grad_norm": 6.627102375030518,
+      "learning_rate": 9.157348061512727e-05,
+      "loss": 4.8802,
+      "step": 166
+    },
+    {
+      "epoch": 0.36885698509110987,
+      "grad_norm": 7.070224761962891,
+      "learning_rate": 9.1323905507714e-05,
+      "loss": 5.1596,
+      "step": 167
+    },
+    {
+      "epoch": 0.37106570955273327,
+      "grad_norm": 7.091405868530273,
+      "learning_rate": 9.107103875602459e-05,
+      "loss": 4.9509,
+      "step": 168
+    },
+    {
+      "epoch": 0.3732744340143567,
+      "grad_norm": 7.415458679199219,
+      "learning_rate": 9.08149005020903e-05,
+      "loss": 4.395,
+      "step": 169
+    },
+    {
+      "epoch": 0.3754831584759801,
+      "grad_norm": 7.86855936050415,
+      "learning_rate": 9.055551114853295e-05,
+      "loss": 5.0404,
+      "step": 170
+    },
+    {
+      "epoch": 0.37769188293760353,
+      "grad_norm": 7.8068718910217285,
+      "learning_rate": 9.02928913569399e-05,
+      "loss": 4.9543,
+      "step": 171
+    },
+    {
+      "epoch": 0.37990060739922693,
+      "grad_norm": 8.077149391174316,
+      "learning_rate": 9.002706204621803e-05,
+      "loss": 4.5707,
+      "step": 172
+    },
+    {
+      "epoch": 0.38210933186085033,
+      "grad_norm": 7.964366912841797,
+      "learning_rate": 8.975804439092765e-05,
+      "loss": 4.6822,
+      "step": 173
+    },
+    {
+      "epoch": 0.3843180563224738,
+      "grad_norm": 8.550042152404785,
+      "learning_rate": 8.94858598195958e-05,
+      "loss": 4.664,
+      "step": 174
+    },
+    {
+      "epoch": 0.3865267807840972,
+      "grad_norm": 8.32715129852295,
+      "learning_rate": 8.921053001300928e-05,
+      "loss": 4.5238,
+      "step": 175
+    },
+    {
+      "epoch": 0.3887355052457206,
+      "grad_norm": 8.401163101196289,
+      "learning_rate": 8.893207690248776e-05,
+      "loss": 5.061,
+      "step": 176
+    },
+    {
+      "epoch": 0.390944229707344,
+      "grad_norm": 8.160713195800781,
+      "learning_rate": 8.865052266813685e-05,
+      "loss": 4.4293,
+      "step": 177
+    },
+    {
+      "epoch": 0.3931529541689674,
+      "grad_norm": 8.723557472229004,
+      "learning_rate": 8.836588973708129e-05,
+      "loss": 4.7296,
+      "step": 178
+    },
+    {
+      "epoch": 0.39536167863059085,
+      "grad_norm": 8.8028564453125,
+      "learning_rate": 8.807820078167856e-05,
+      "loss": 4.9615,
+      "step": 179
+    },
+    {
+      "epoch": 0.39757040309221425,
+      "grad_norm": 8.326678276062012,
+      "learning_rate": 8.778747871771292e-05,
+      "loss": 4.9342,
+      "step": 180
+    },
+    {
+      "epoch": 0.39977912755383765,
+      "grad_norm": 9.141505241394043,
+      "learning_rate": 8.749374670257005e-05,
+      "loss": 5.0855,
+      "step": 181
+    },
+    {
+      "epoch": 0.40198785201546106,
+      "grad_norm": 8.820999145507812,
+      "learning_rate": 8.719702813339248e-05,
+      "loss": 5.3793,
+      "step": 182
+    },
+    {
+      "epoch": 0.40419657647708446,
+      "grad_norm": 8.986492156982422,
+      "learning_rate": 8.689734664521583e-05,
+      "loss": 5.0518,
+      "step": 183
+    },
+    {
+      "epoch": 0.4064053009387079,
+      "grad_norm": 8.636345863342285,
+      "learning_rate": 8.659472610908627e-05,
+      "loss": 5.2209,
+      "step": 184
+    },
+    {
+      "epoch": 0.4086140254003313,
+      "grad_norm": 8.594338417053223,
+      "learning_rate": 8.628919063015898e-05,
+      "loss": 5.1603,
+      "step": 185
+    },
+    {
+      "epoch": 0.4108227498619547,
+      "grad_norm": 8.710060119628906,
+      "learning_rate": 8.598076454577814e-05,
+      "loss": 5.2937,
+      "step": 186
+    },
+    {
+      "epoch": 0.4130314743235781,
+      "grad_norm": 8.30070972442627,
+      "learning_rate": 8.566947242353827e-05,
+      "loss": 4.9801,
+      "step": 187
+    },
+    {
+      "epoch": 0.4152401987852015,
+      "grad_norm": 8.696635246276855,
+      "learning_rate": 8.535533905932738e-05,
+      "loss": 4.7529,
+      "step": 188
+    },
+    {
+      "epoch": 0.417448923246825,
+      "grad_norm": 8.520990371704102,
+      "learning_rate": 8.503838947535179e-05,
+      "loss": 5.344,
+      "step": 189
+    },
+    {
+      "epoch": 0.4196576477084484,
+      "grad_norm": 8.845208168029785,
+      "learning_rate": 8.471864891814304e-05,
+      "loss": 4.7962,
+      "step": 190
+    },
+    {
+      "epoch": 0.4218663721700718,
+      "grad_norm": 8.896949768066406,
+      "learning_rate": 8.439614285654689e-05,
+      "loss": 5.1538,
+      "step": 191
+    },
+    {
+      "epoch": 0.4240750966316952,
+      "grad_norm": 8.677151679992676,
+      "learning_rate": 8.407089697969457e-05,
+      "loss": 5.0412,
+      "step": 192
+    },
+    {
+      "epoch": 0.4262838210933186,
+      "grad_norm": 9.09294319152832,
+      "learning_rate": 8.374293719495651e-05,
+      "loss": 4.8806,
+      "step": 193
+    },
+    {
+      "epoch": 0.42849254555494204,
+      "grad_norm": 8.677190780639648,
+      "learning_rate": 8.341228962587882e-05,
+      "loss": 4.5531,
+      "step": 194
+    },
+    {
+      "epoch": 0.43070127001656544,
+      "grad_norm": 9.5293550491333,
+      "learning_rate": 8.307898061010221e-05,
+      "loss": 5.1039,
+      "step": 195
+    },
+    {
+      "epoch": 0.43290999447818884,
+      "grad_norm": 9.922966957092285,
+      "learning_rate": 8.274303669726426e-05,
+      "loss": 5.0978,
+      "step": 196
+    },
+    {
+      "epoch": 0.43511871893981224,
+      "grad_norm": 9.41407585144043,
+      "learning_rate": 8.240448464688453e-05,
+      "loss": 4.5112,
+      "step": 197
+    },
+    {
+      "epoch": 0.43732744340143564,
+      "grad_norm": 10.871786117553711,
+      "learning_rate": 8.206335142623305e-05,
+      "loss": 4.7271,
+      "step": 198
+    },
+    {
+      "epoch": 0.4395361678630591,
+      "grad_norm": 11.003656387329102,
+      "learning_rate": 8.171966420818228e-05,
+      "loss": 4.8793,
+      "step": 199
+    },
+    {
+      "epoch": 0.4417448923246825,
+      "grad_norm": 12.43280029296875,
+      "learning_rate": 8.13734503690426e-05,
+      "loss": 5.1866,
+      "step": 200
+    },
+    {
+      "epoch": 0.4439536167863059,
+      "grad_norm": 5.124542713165283,
+      "learning_rate": 8.102473748638173e-05,
+      "loss": 3.8602,
+      "step": 201
+    },
+    {
+      "epoch": 0.4461623412479293,
+      "grad_norm": 7.500517845153809,
+      "learning_rate": 8.067355333682798e-05,
+      "loss": 4.7929,
+      "step": 202
+    },
+    {
+      "epoch": 0.4483710657095527,
+      "grad_norm": 7.736271858215332,
+      "learning_rate": 8.031992589385777e-05,
+      "loss": 4.9299,
+      "step": 203
+    },
+    {
+      "epoch": 0.45057979017117616,
+      "grad_norm": 8.110419273376465,
+      "learning_rate": 7.996388332556735e-05,
+      "loss": 4.9557,
+      "step": 204
+    },
+    {
+      "epoch": 0.45278851463279957,
+      "grad_norm": 6.666763782501221,
+      "learning_rate": 7.960545399242916e-05,
+      "loss": 4.5717,
+      "step": 205
+    },
+    {
+      "epoch": 0.45499723909442297,
+      "grad_norm": 5.63977575302124,
+      "learning_rate": 7.924466644503265e-05,
+      "loss": 4.2633,
+      "step": 206
+    },
+    {
+      "epoch": 0.45720596355604637,
+      "grad_norm": 5.997104644775391,
+      "learning_rate": 7.88815494218103e-05,
+      "loss": 4.6012,
+      "step": 207
+    },
+    {
+      "epoch": 0.45941468801766977,
+      "grad_norm": 5.42753267288208,
+      "learning_rate": 7.85161318467482e-05,
+      "loss": 4.9035,
+      "step": 208
+    },
+    {
+      "epoch": 0.4616234124792932,
+      "grad_norm": 5.311511039733887,
+      "learning_rate": 7.814844282708239e-05,
+      "loss": 4.4711,
+      "step": 209
+    },
+    {
+      "epoch": 0.46383213694091663,
+      "grad_norm": 5.2506256103515625,
+      "learning_rate": 7.777851165098012e-05,
+      "loss": 4.6818,
+      "step": 210
+    },
+    {
+      "epoch": 0.46604086140254003,
+      "grad_norm": 5.2805280685424805,
+      "learning_rate": 7.7406367785207e-05,
+      "loss": 4.9481,
+      "step": 211
+    },
+    {
+      "epoch": 0.46824958586416343,
+      "grad_norm": 5.382137298583984,
+      "learning_rate": 7.703204087277988e-05,
+      "loss": 4.5646,
+      "step": 212
+    },
+    {
+      "epoch": 0.47045831032578683,
+      "grad_norm": 5.792073726654053,
+      "learning_rate": 7.665556073060552e-05,
+      "loss": 4.8119,
+      "step": 213
+    },
+    {
+      "epoch": 0.4726670347874103,
+      "grad_norm": 6.007907867431641,
+      "learning_rate": 7.627695734710564e-05,
+      "loss": 4.687,
+      "step": 214
+    },
+    {
+      "epoch": 0.4748757592490337,
+      "grad_norm": 5.8332109451293945,
+      "learning_rate": 7.589626087982816e-05,
+      "loss": 4.1969,
+      "step": 215
+    },
+    {
+      "epoch": 0.4770844837106571,
+      "grad_norm": 6.367125511169434,
+      "learning_rate": 7.5513501653045e-05,
+      "loss": 4.3616,
+      "step": 216
+    },
+    {
+      "epoch": 0.4792932081722805,
+      "grad_norm": 6.897268772125244,
+      "learning_rate": 7.512871015533658e-05,
+      "loss": 4.9368,
+      "step": 217
+    },
+    {
+      "epoch": 0.4815019326339039,
+      "grad_norm": 7.102772235870361,
+      "learning_rate": 7.474191703716339e-05,
+      "loss": 4.8659,
+      "step": 218
+    },
+    {
+      "epoch": 0.48371065709552735,
+      "grad_norm": 7.861514568328857,
+      "learning_rate": 7.435315310842436e-05,
+      "loss": 4.5714,
+      "step": 219
+    },
+    {
+      "epoch": 0.48591938155715075,
+      "grad_norm": 8.072993278503418,
+      "learning_rate": 7.396244933600285e-05,
+      "loss": 4.668,
+      "step": 220
+    },
+    {
+      "epoch": 0.48812810601877415,
+      "grad_norm": 8.46921443939209,
+      "learning_rate": 7.35698368412999e-05,
+      "loss": 4.932,
+      "step": 221
+    },
+    {
+      "epoch": 0.49033683048039756,
+      "grad_norm": 7.927035331726074,
+      "learning_rate": 7.317534689775528e-05,
+      "loss": 4.6539,
+      "step": 222
+    },
+    {
+      "epoch": 0.49254555494202096,
+      "grad_norm": 8.272632598876953,
+      "learning_rate": 7.277901092835649e-05,
+      "loss": 4.7136,
+      "step": 223
+    },
+    {
+      "epoch": 0.4947542794036444,
+      "grad_norm": 8.767943382263184,
+      "learning_rate": 7.238086050313563e-05,
+      "loss": 4.7382,
+      "step": 224
+    },
+    {
+      "epoch": 0.4969630038652678,
+      "grad_norm": 8.93777084350586,
+      "learning_rate": 7.198092733665487e-05,
+      "loss": 5.0617,
+      "step": 225
+    },
+    {
+      "epoch": 0.4991717283268912,
+      "grad_norm": 8.695636749267578,
+      "learning_rate": 7.157924328548002e-05,
+      "loss": 4.9294,
+      "step": 226
+    },
+    {
+      "epoch": 0.5013804527885146,
+      "grad_norm": 8.272305488586426,
+      "learning_rate": 7.117584034564329e-05,
+      "loss": 4.5938,
+      "step": 227
+    },
+    {
+      "epoch": 0.5035891772501381,
+      "grad_norm": 8.093917846679688,
+      "learning_rate": 7.077075065009433e-05,
+      "loss": 4.6757,
+      "step": 228
+    },
+    {
+      "epoch": 0.5057979017117614,
+      "grad_norm": 8.725174903869629,
+      "learning_rate": 7.036400646614094e-05,
+      "loss": 4.9074,
+      "step": 229
+    },
+    {
+      "epoch": 0.5080066261733849,
+      "grad_norm": 8.211441040039062,
+      "learning_rate": 6.995564019287871e-05,
+      "loss": 4.7963,
+      "step": 230
+    },
+    {
+      "epoch": 0.5102153506350083,
+      "grad_norm": 8.903889656066895,
+      "learning_rate": 6.954568435861033e-05,
+      "loss": 4.9038,
+      "step": 231
+    },
+    {
+      "epoch": 0.5124240750966317,
+      "grad_norm": 8.311030387878418,
+      "learning_rate": 6.91341716182545e-05,
+      "loss": 4.8634,
+      "step": 232
+    },
+    {
+      "epoch": 0.5146327995582551,
+      "grad_norm": 8.491503715515137,
+      "learning_rate": 6.872113475074486e-05,
+      "loss": 5.043,
+      "step": 233
+    },
+    {
+      "epoch": 0.5168415240198785,
+      "grad_norm": 8.99098014831543,
+      "learning_rate": 6.830660665641898e-05,
+      "loss": 5.0784,
+      "step": 234
+    },
+    {
+      "epoch": 0.5190502484815019,
+      "grad_norm": 8.576278686523438,
+      "learning_rate": 6.789062035439767e-05,
+      "loss": 4.4813,
+      "step": 235
+    },
+    {
+      "epoch": 0.5212589729431254,
+      "grad_norm": 8.29208755493164,
+      "learning_rate": 6.747320897995493e-05,
+      "loss": 4.9514,
+      "step": 236
+    },
+    {
+      "epoch": 0.5234676974047487,
+      "grad_norm": 9.072782516479492,
+      "learning_rate": 6.705440578187842e-05,
+      "loss": 5.0796,
+      "step": 237
+    },
+    {
+      "epoch": 0.5256764218663722,
+      "grad_norm": 9.66883373260498,
+      "learning_rate": 6.663424411982121e-05,
+      "loss": 4.745,
+      "step": 238
+    },
+    {
+      "epoch": 0.5278851463279955,
+      "grad_norm": 8.428522109985352,
+      "learning_rate": 6.621275746164437e-05,
+      "loss": 4.599,
+      "step": 239
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 452,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 239,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.04116284864594e+16,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdc92be7ecdabf4679459fab4b3bc292ff96903e6d3d8ee9dd7c489382bc9a46
+size 6840