Upload folder using huggingface_hub

README.md

@@ -0,0 +1,40 @@
+---
+tags:
+- autotrain
+- text-generation
+widget:
+- text: "I love AutoTrain because "
+license: other
+---
+
+# Model Trained Using AutoTrain
+
+This model was trained using AutoTrain. For more information, please visit [AutoTrain](https://hf.co/docs/autotrain).
+
+# Usage
+
+```python
+
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_path = "PATH_TO_THIS_REPO"
+
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(
+    model_path,
+    device_map="auto",
+    torch_dtype='auto'
+).eval()
+
+# Prompt content: "hi"
+messages = [
+    {"role": "user", "content": "hi"}
+]
+
+input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
+output_ids = model.generate(input_ids.to('cuda'))
+response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)
+
+# Model response: "Hello! How can I assist you today?"
+print(response)
+```

checkpoint-450/README.md

@@ -0,0 +1,204 @@
+---
+library_name: peft
+base_model: HuggingFaceH4/zephyr-7b-beta
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+### Framework versions
+
+- PEFT 0.7.1

checkpoint-450/adapter_config.json

@@ -0,0 +1,26 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "HuggingFaceH4/zephyr-7b-beta",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM"
+}

checkpoint-450/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:876481d17a63a6ca9964e29bc86e51a959f392a78c366674375bb7bd87da0836
+size 27280152

checkpoint-450/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:576409de17d3382752d1b47065e1f325980a0d176dfbc41e8ce6523780dfeb57
+size 54633978

checkpoint-450/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:049c26b844b79121ddd8379f7f69194e63f6fbf6aa007eeac0c66f17eebb8893
+size 888

checkpoint-450/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1503058efffbdc3efefc668db5de2856ae97543b8639eb2ec61c5d8b43e911bd
+size 14244

checkpoint-450/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4d4b34e7afaf300867939795c9615a9f6f382d574e467530e28ed648f1055fd
+size 1064

checkpoint-450/special_tokens_map.json

@@ -0,0 +1,35 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-450/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-450/tokenizer_config.json

@@ -0,0 +1,48 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 2048,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "left",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

checkpoint-450/trainer_state.json

@@ -0,0 +1,2721 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 50.0,
+  "eval_steps": 500,
+  "global_step": 450,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.11,
+      "learning_rate": 0.00022222222222222223,
+      "loss": 1.7682,
+      "step": 1
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 0.00044444444444444447,
+      "loss": 1.9883,
+      "step": 2
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 0.0006666666666666666,
+      "loss": 1.7947,
+      "step": 3
+    },
+    {
+      "epoch": 0.44,
+      "learning_rate": 0.0008888888888888889,
+      "loss": 1.9388,
+      "step": 4
+    },
+    {
+      "epoch": 0.56,
+      "learning_rate": 0.0011111111111111111,
+      "loss": 1.8301,
+      "step": 5
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 0.0011111111111111111,
+      "loss": 1.7121,
+      "step": 6
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 0.0011111111111111111,
+      "loss": 1.7248,
+      "step": 7
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 0.0013333333333333333,
+      "loss": 1.6582,
+      "step": 8
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 0.0013333333333333333,
+      "loss": 2.0618,
+      "step": 9
+    },
+    {
+      "epoch": 1.11,
+      "learning_rate": 0.0013333333333333333,
+      "loss": 2.0288,
+      "step": 10
+    },
+    {
+      "epoch": 1.22,
+      "learning_rate": 0.0013333333333333333,
+      "loss": 6.2236,
+      "step": 11
+    },
+    {
+      "epoch": 1.33,
+      "learning_rate": 0.0013333333333333333,
+      "loss": 4.7077,
+      "step": 12
+    },
+    {
+      "epoch": 1.44,
+      "learning_rate": 0.0013333333333333333,
+      "loss": 2.8244,
+      "step": 13
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 0.0013333333333333333,
+      "loss": 3.6065,
+      "step": 14
+    },
+    {
+      "epoch": 1.67,
+      "learning_rate": 0.0015555555555555557,
+      "loss": 2.9129,
+      "step": 15
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 0.0017777777777777779,
+      "loss": 1.5032,
+      "step": 16
+    },
+    {
+      "epoch": 1.89,
+      "learning_rate": 0.002,
+      "loss": 1.2412,
+      "step": 17
+    },
+    {
+      "epoch": 2.0,
+      "learning_rate": 0.0022222222222222222,
+      "loss": 1.7111,
+      "step": 18
+    },
+    {
+      "epoch": 2.11,
+      "learning_rate": 0.0024444444444444444,
+      "loss": 1.6141,
+      "step": 19
+    },
+    {
+      "epoch": 2.22,
+      "learning_rate": 0.0026666666666666666,
+      "loss": 1.6745,
+      "step": 20
+    },
+    {
+      "epoch": 2.33,
+      "learning_rate": 0.0028888888888888888,
+      "loss": 2.4298,
+      "step": 21
+    },
+    {
+      "epoch": 2.44,
+      "learning_rate": 0.0031111111111111114,
+      "loss": 1.6993,
+      "step": 22
+    },
+    {
+      "epoch": 2.56,
+      "learning_rate": 0.003333333333333333,
+      "loss": 4.6999,
+      "step": 23
+    },
+    {
+      "epoch": 2.67,
+      "learning_rate": 0.0035555555555555557,
+      "loss": 3.5699,
+      "step": 24
+    },
+    {
+      "epoch": 2.78,
+      "learning_rate": 0.003777777777777778,
+      "loss": 8.7543,
+      "step": 25
+    },
+    {
+      "epoch": 2.89,
+      "learning_rate": 0.004,
+      "loss": 9.8707,
+      "step": 26
+    },
+    {
+      "epoch": 3.0,
+      "learning_rate": 0.004222222222222223,
+      "loss": 10.1086,
+      "step": 27
+    },
+    {
+      "epoch": 3.11,
+      "learning_rate": 0.0044444444444444444,
+      "loss": 9.2775,
+      "step": 28
+    },
+    {
+      "epoch": 3.22,
+      "learning_rate": 0.004666666666666667,
+      "loss": 7.8785,
+      "step": 29
+    },
+    {
+      "epoch": 3.33,
+      "learning_rate": 0.004888888888888889,
+      "loss": 7.472,
+      "step": 30
+    },
+    {
+      "epoch": 3.44,
+      "learning_rate": 0.0051111111111111105,
+      "loss": 7.2495,
+      "step": 31
+    },
+    {
+      "epoch": 3.56,
+      "learning_rate": 0.005333333333333333,
+      "loss": 13.0327,
+      "step": 32
+    },
+    {
+      "epoch": 3.67,
+      "learning_rate": 0.005555555555555556,
+      "loss": 15.2515,
+      "step": 33
+    },
+    {
+      "epoch": 3.78,
+      "learning_rate": 0.0057777777777777775,
+      "loss": 12.4308,
+      "step": 34
+    },
+    {
+      "epoch": 3.89,
+      "learning_rate": 0.006,
+      "loss": 11.2797,
+      "step": 35
+    },
+    {
+      "epoch": 4.0,
+      "learning_rate": 0.006222222222222223,
+      "loss": 9.5219,
+      "step": 36
+    },
+    {
+      "epoch": 4.11,
+      "learning_rate": 0.006444444444444445,
+      "loss": 7.6947,
+      "step": 37
+    },
+    {
+      "epoch": 4.22,
+      "learning_rate": 0.006444444444444445,
+      "loss": 11.9818,
+      "step": 38
+    },
+    {
+      "epoch": 4.33,
+      "learning_rate": 0.006444444444444445,
+      "loss": 14.0646,
+      "step": 39
+    },
+    {
+      "epoch": 4.44,
+      "learning_rate": 0.006666666666666666,
+      "loss": 12.282,
+      "step": 40
+    },
+    {
+      "epoch": 4.56,
+      "learning_rate": 0.006888888888888889,
+      "loss": 10.8636,
+      "step": 41
+    },
+    {
+      "epoch": 4.67,
+      "learning_rate": 0.0071111111111111115,
+      "loss": 8.8291,
+      "step": 42
+    },
+    {
+      "epoch": 4.78,
+      "learning_rate": 0.007333333333333333,
+      "loss": 8.4366,
+      "step": 43
+    },
+    {
+      "epoch": 4.89,
+      "learning_rate": 0.007555555555555556,
+      "loss": 8.9278,
+      "step": 44
+    },
+    {
+      "epoch": 5.0,
+      "learning_rate": 0.007777777777777778,
+      "loss": 11.5827,
+      "step": 45
+    },
+    {
+      "epoch": 5.11,
+      "learning_rate": 0.008,
+      "loss": 8.8135,
+      "step": 46
+    },
+    {
+      "epoch": 5.22,
+      "learning_rate": 0.008222222222222223,
+      "loss": 7.5243,
+      "step": 47
+    },
+    {
+      "epoch": 5.33,
+      "learning_rate": 0.008444444444444445,
+      "loss": 8.1523,
+      "step": 48
+    },
+    {
+      "epoch": 5.44,
+      "learning_rate": 0.008666666666666668,
+      "loss": 8.2508,
+      "step": 49
+    },
+    {
+      "epoch": 5.56,
+      "learning_rate": 0.008888888888888889,
+      "loss": 7.961,
+      "step": 50
+    },
+    {
+      "epoch": 5.67,
+      "learning_rate": 0.009111111111111111,
+      "loss": 7.3086,
+      "step": 51
+    },
+    {
+      "epoch": 5.78,
+      "learning_rate": 0.009333333333333334,
+      "loss": 7.3684,
+      "step": 52
+    },
+    {
+      "epoch": 5.89,
+      "learning_rate": 0.009555555555555557,
+      "loss": 7.3783,
+      "step": 53
+    },
+    {
+      "epoch": 6.0,
+      "learning_rate": 0.009777777777777778,
+      "loss": 9.3611,
+      "step": 54
+    },
+    {
+      "epoch": 6.11,
+      "learning_rate": 0.01,
+      "loss": 12.1977,
+      "step": 55
+    },
+    {
+      "epoch": 6.22,
+      "learning_rate": 0.009975308641975308,
+      "loss": 7.2867,
+      "step": 56
+    },
+    {
+      "epoch": 6.33,
+      "learning_rate": 0.009950617283950617,
+      "loss": 8.0552,
+      "step": 57
+    },
+    {
+      "epoch": 6.44,
+      "learning_rate": 0.009925925925925927,
+      "loss": 7.7156,
+      "step": 58
+    },
+    {
+      "epoch": 6.56,
+      "learning_rate": 0.009901234567901235,
+      "loss": 7.3561,
+      "step": 59
+    },
+    {
+      "epoch": 6.67,
+      "learning_rate": 0.009876543209876543,
+      "loss": 7.4133,
+      "step": 60
+    },
+    {
+      "epoch": 6.78,
+      "learning_rate": 0.009851851851851851,
+      "loss": 16.2833,
+      "step": 61
+    },
+    {
+      "epoch": 6.89,
+      "learning_rate": 0.009827160493827161,
+      "loss": 8.633,
+      "step": 62
+    },
+    {
+      "epoch": 7.0,
+      "learning_rate": 0.00980246913580247,
+      "loss": 6.6628,
+      "step": 63
+    },
+    {
+      "epoch": 7.11,
+      "learning_rate": 0.009777777777777778,
+      "loss": 6.9888,
+      "step": 64
+    },
+    {
+      "epoch": 7.22,
+      "learning_rate": 0.009753086419753086,
+      "loss": 7.3082,
+      "step": 65
+    },
+    {
+      "epoch": 7.33,
+      "learning_rate": 0.009728395061728396,
+      "loss": 7.2472,
+      "step": 66
+    },
+    {
+      "epoch": 7.44,
+      "learning_rate": 0.009703703703703704,
+      "loss": 7.0683,
+      "step": 67
+    },
+    {
+      "epoch": 7.56,
+      "learning_rate": 0.009679012345679012,
+      "loss": 7.1155,
+      "step": 68
+    },
+    {
+      "epoch": 7.67,
+      "learning_rate": 0.009654320987654322,
+      "loss": 6.8406,
+      "step": 69
+    },
+    {
+      "epoch": 7.78,
+      "learning_rate": 0.009629629629629629,
+      "loss": 7.1108,
+      "step": 70
+    },
+    {
+      "epoch": 7.89,
+      "learning_rate": 0.009604938271604939,
+      "loss": 6.7208,
+      "step": 71
+    },
+    {
+      "epoch": 8.0,
+      "learning_rate": 0.009580246913580247,
+      "loss": 6.7704,
+      "step": 72
+    },
+    {
+      "epoch": 8.11,
+      "learning_rate": 0.009555555555555557,
+      "loss": 6.8468,
+      "step": 73
+    },
+    {
+      "epoch": 8.22,
+      "learning_rate": 0.009530864197530865,
+      "loss": 6.9493,
+      "step": 74
+    },
+    {
+      "epoch": 8.33,
+      "learning_rate": 0.009506172839506173,
+      "loss": 7.0246,
+      "step": 75
+    },
+    {
+      "epoch": 8.44,
+      "learning_rate": 0.009481481481481481,
+      "loss": 6.7802,
+      "step": 76
+    },
+    {
+      "epoch": 8.56,
+      "learning_rate": 0.00945679012345679,
+      "loss": 6.8637,
+      "step": 77
+    },
+    {
+      "epoch": 8.67,
+      "learning_rate": 0.0094320987654321,
+      "loss": 6.8141,
+      "step": 78
+    },
+    {
+      "epoch": 8.78,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 79
+    },
+    {
+      "epoch": 8.89,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 80
+    },
+    {
+      "epoch": 9.0,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 81
+    },
+    {
+      "epoch": 9.11,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 82
+    },
+    {
+      "epoch": 9.22,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 83
+    },
+    {
+      "epoch": 9.33,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 84
+    },
+    {
+      "epoch": 9.44,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 85
+    },
+    {
+      "epoch": 9.56,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 86
+    },
+    {
+      "epoch": 9.67,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 87
+    },
+    {
+      "epoch": 9.78,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 88
+    },
+    {
+      "epoch": 9.89,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 89
+    },
+    {
+      "epoch": 10.0,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 90
+    },
+    {
+      "epoch": 10.11,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 91
+    },
+    {
+      "epoch": 10.22,
+      "learning_rate": 0.0094320987654321,
+      "loss": 0.0,
+      "step": 92
+    },
+    {
+      "epoch": 10.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 7.0349,
+      "step": 93
+    },
+    {
+      "epoch": 10.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 94
+    },
+    {
+      "epoch": 10.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 95
+    },
+    {
+      "epoch": 10.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 96
+    },
+    {
+      "epoch": 10.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 97
+    },
+    {
+      "epoch": 10.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 98
+    },
+    {
+      "epoch": 11.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 99
+    },
+    {
+      "epoch": 11.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 100
+    },
+    {
+      "epoch": 11.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 101
+    },
+    {
+      "epoch": 11.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 102
+    },
+    {
+      "epoch": 11.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 103
+    },
+    {
+      "epoch": 11.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 104
+    },
+    {
+      "epoch": 11.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 105
+    },
+    {
+      "epoch": 11.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 106
+    },
+    {
+      "epoch": 11.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 107
+    },
+    {
+      "epoch": 12.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 108
+    },
+    {
+      "epoch": 12.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 109
+    },
+    {
+      "epoch": 12.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 110
+    },
+    {
+      "epoch": 12.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 111
+    },
+    {
+      "epoch": 12.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 112
+    },
+    {
+      "epoch": 12.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 113
+    },
+    {
+      "epoch": 12.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 114
+    },
+    {
+      "epoch": 12.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 115
+    },
+    {
+      "epoch": 12.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 116
+    },
+    {
+      "epoch": 13.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 117
+    },
+    {
+      "epoch": 13.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 118
+    },
+    {
+      "epoch": 13.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 119
+    },
+    {
+      "epoch": 13.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 120
+    },
+    {
+      "epoch": 13.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 121
+    },
+    {
+      "epoch": 13.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 122
+    },
+    {
+      "epoch": 13.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 123
+    },
+    {
+      "epoch": 13.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 124
+    },
+    {
+      "epoch": 13.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 125
+    },
+    {
+      "epoch": 14.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 126
+    },
+    {
+      "epoch": 14.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 127
+    },
+    {
+      "epoch": 14.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 128
+    },
+    {
+      "epoch": 14.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 129
+    },
+    {
+      "epoch": 14.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 130
+    },
+    {
+      "epoch": 14.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 131
+    },
+    {
+      "epoch": 14.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 132
+    },
+    {
+      "epoch": 14.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 133
+    },
+    {
+      "epoch": 14.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 134
+    },
+    {
+      "epoch": 15.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 135
+    },
+    {
+      "epoch": 15.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 136
+    },
+    {
+      "epoch": 15.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 137
+    },
+    {
+      "epoch": 15.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 138
+    },
+    {
+      "epoch": 15.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 139
+    },
+    {
+      "epoch": 15.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 140
+    },
+    {
+      "epoch": 15.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 141
+    },
+    {
+      "epoch": 15.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 142
+    },
+    {
+      "epoch": 15.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 143
+    },
+    {
+      "epoch": 16.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 144
+    },
+    {
+      "epoch": 16.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 145
+    },
+    {
+      "epoch": 16.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 146
+    },
+    {
+      "epoch": 16.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 147
+    },
+    {
+      "epoch": 16.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 148
+    },
+    {
+      "epoch": 16.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 149
+    },
+    {
+      "epoch": 16.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 150
+    },
+    {
+      "epoch": 16.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 151
+    },
+    {
+      "epoch": 16.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 152
+    },
+    {
+      "epoch": 17.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 153
+    },
+    {
+      "epoch": 17.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 154
+    },
+    {
+      "epoch": 17.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 155
+    },
+    {
+      "epoch": 17.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 156
+    },
+    {
+      "epoch": 17.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 157
+    },
+    {
+      "epoch": 17.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 158
+    },
+    {
+      "epoch": 17.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 159
+    },
+    {
+      "epoch": 17.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 160
+    },
+    {
+      "epoch": 17.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 161
+    },
+    {
+      "epoch": 18.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 162
+    },
+    {
+      "epoch": 18.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 163
+    },
+    {
+      "epoch": 18.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 164
+    },
+    {
+      "epoch": 18.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 165
+    },
+    {
+      "epoch": 18.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 166
+    },
+    {
+      "epoch": 18.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 167
+    },
+    {
+      "epoch": 18.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 168
+    },
+    {
+      "epoch": 18.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 169
+    },
+    {
+      "epoch": 18.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 170
+    },
+    {
+      "epoch": 19.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 171
+    },
+    {
+      "epoch": 19.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 172
+    },
+    {
+      "epoch": 19.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 173
+    },
+    {
+      "epoch": 19.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 174
+    },
+    {
+      "epoch": 19.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 175
+    },
+    {
+      "epoch": 19.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 176
+    },
+    {
+      "epoch": 19.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 177
+    },
+    {
+      "epoch": 19.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 178
+    },
+    {
+      "epoch": 19.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 179
+    },
+    {
+      "epoch": 20.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 180
+    },
+    {
+      "epoch": 20.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 181
+    },
+    {
+      "epoch": 20.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 182
+    },
+    {
+      "epoch": 20.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 183
+    },
+    {
+      "epoch": 20.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 184
+    },
+    {
+      "epoch": 20.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 185
+    },
+    {
+      "epoch": 20.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 186
+    },
+    {
+      "epoch": 20.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 187
+    },
+    {
+      "epoch": 20.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 188
+    },
+    {
+      "epoch": 21.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 189
+    },
+    {
+      "epoch": 21.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 190
+    },
+    {
+      "epoch": 21.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 191
+    },
+    {
+      "epoch": 21.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 192
+    },
+    {
+      "epoch": 21.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 193
+    },
+    {
+      "epoch": 21.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 194
+    },
+    {
+      "epoch": 21.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 195
+    },
+    {
+      "epoch": 21.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 196
+    },
+    {
+      "epoch": 21.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 197
+    },
+    {
+      "epoch": 22.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 198
+    },
+    {
+      "epoch": 22.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 199
+    },
+    {
+      "epoch": 22.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 200
+    },
+    {
+      "epoch": 22.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 201
+    },
+    {
+      "epoch": 22.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 202
+    },
+    {
+      "epoch": 22.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 203
+    },
+    {
+      "epoch": 22.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 204
+    },
+    {
+      "epoch": 22.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 205
+    },
+    {
+      "epoch": 22.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 206
+    },
+    {
+      "epoch": 23.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 207
+    },
+    {
+      "epoch": 23.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 208
+    },
+    {
+      "epoch": 23.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 209
+    },
+    {
+      "epoch": 23.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 210
+    },
+    {
+      "epoch": 23.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 211
+    },
+    {
+      "epoch": 23.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 212
+    },
+    {
+      "epoch": 23.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 213
+    },
+    {
+      "epoch": 23.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 214
+    },
+    {
+      "epoch": 23.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 215
+    },
+    {
+      "epoch": 24.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 216
+    },
+    {
+      "epoch": 24.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 217
+    },
+    {
+      "epoch": 24.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 218
+    },
+    {
+      "epoch": 24.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 219
+    },
+    {
+      "epoch": 24.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 220
+    },
+    {
+      "epoch": 24.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 221
+    },
+    {
+      "epoch": 24.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 222
+    },
+    {
+      "epoch": 24.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 223
+    },
+    {
+      "epoch": 24.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 224
+    },
+    {
+      "epoch": 25.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 225
+    },
+    {
+      "epoch": 25.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 226
+    },
+    {
+      "epoch": 25.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 227
+    },
+    {
+      "epoch": 25.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 228
+    },
+    {
+      "epoch": 25.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 229
+    },
+    {
+      "epoch": 25.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 230
+    },
+    {
+      "epoch": 25.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 231
+    },
+    {
+      "epoch": 25.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 232
+    },
+    {
+      "epoch": 25.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 233
+    },
+    {
+      "epoch": 26.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 234
+    },
+    {
+      "epoch": 26.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 235
+    },
+    {
+      "epoch": 26.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 236
+    },
+    {
+      "epoch": 26.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 237
+    },
+    {
+      "epoch": 26.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 238
+    },
+    {
+      "epoch": 26.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 239
+    },
+    {
+      "epoch": 26.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 240
+    },
+    {
+      "epoch": 26.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 241
+    },
+    {
+      "epoch": 26.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 242
+    },
+    {
+      "epoch": 27.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 243
+    },
+    {
+      "epoch": 27.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 244
+    },
+    {
+      "epoch": 27.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 245
+    },
+    {
+      "epoch": 27.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 246
+    },
+    {
+      "epoch": 27.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 247
+    },
+    {
+      "epoch": 27.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 248
+    },
+    {
+      "epoch": 27.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 249
+    },
+    {
+      "epoch": 27.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 250
+    },
+    {
+      "epoch": 27.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 251
+    },
+    {
+      "epoch": 28.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 252
+    },
+    {
+      "epoch": 28.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 253
+    },
+    {
+      "epoch": 28.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 254
+    },
+    {
+      "epoch": 28.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 255
+    },
+    {
+      "epoch": 28.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 256
+    },
+    {
+      "epoch": 28.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 257
+    },
+    {
+      "epoch": 28.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 258
+    },
+    {
+      "epoch": 28.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 259
+    },
+    {
+      "epoch": 28.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 260
+    },
+    {
+      "epoch": 29.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 261
+    },
+    {
+      "epoch": 29.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 262
+    },
+    {
+      "epoch": 29.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 263
+    },
+    {
+      "epoch": 29.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 264
+    },
+    {
+      "epoch": 29.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 265
+    },
+    {
+      "epoch": 29.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 266
+    },
+    {
+      "epoch": 29.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 267
+    },
+    {
+      "epoch": 29.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 268
+    },
+    {
+      "epoch": 29.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 269
+    },
+    {
+      "epoch": 30.0,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 270
+    },
+    {
+      "epoch": 30.11,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 271
+    },
+    {
+      "epoch": 30.22,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 272
+    },
+    {
+      "epoch": 30.33,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 273
+    },
+    {
+      "epoch": 30.44,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 274
+    },
+    {
+      "epoch": 30.56,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 275
+    },
+    {
+      "epoch": 30.67,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 276
+    },
+    {
+      "epoch": 30.78,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 277
+    },
+    {
+      "epoch": 30.89,
+      "learning_rate": 0.009407407407407408,
+      "loss": 0.0,
+      "step": 278
+    },
+    {
+      "epoch": 31.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 6.9362,
+      "step": 279
+    },
+    {
+      "epoch": 31.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 280
+    },
+    {
+      "epoch": 31.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 281
+    },
+    {
+      "epoch": 31.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 282
+    },
+    {
+      "epoch": 31.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 283
+    },
+    {
+      "epoch": 31.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 284
+    },
+    {
+      "epoch": 31.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 285
+    },
+    {
+      "epoch": 31.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 286
+    },
+    {
+      "epoch": 31.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 287
+    },
+    {
+      "epoch": 32.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 288
+    },
+    {
+      "epoch": 32.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 289
+    },
+    {
+      "epoch": 32.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 290
+    },
+    {
+      "epoch": 32.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 291
+    },
+    {
+      "epoch": 32.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 292
+    },
+    {
+      "epoch": 32.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 293
+    },
+    {
+      "epoch": 32.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 294
+    },
+    {
+      "epoch": 32.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 295
+    },
+    {
+      "epoch": 32.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 296
+    },
+    {
+      "epoch": 33.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 297
+    },
+    {
+      "epoch": 33.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 298
+    },
+    {
+      "epoch": 33.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 299
+    },
+    {
+      "epoch": 33.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 300
+    },
+    {
+      "epoch": 33.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 301
+    },
+    {
+      "epoch": 33.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 302
+    },
+    {
+      "epoch": 33.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 303
+    },
+    {
+      "epoch": 33.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 304
+    },
+    {
+      "epoch": 33.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 305
+    },
+    {
+      "epoch": 34.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 306
+    },
+    {
+      "epoch": 34.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 307
+    },
+    {
+      "epoch": 34.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 308
+    },
+    {
+      "epoch": 34.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 309
+    },
+    {
+      "epoch": 34.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 310
+    },
+    {
+      "epoch": 34.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 311
+    },
+    {
+      "epoch": 34.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 312
+    },
+    {
+      "epoch": 34.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 313
+    },
+    {
+      "epoch": 34.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 314
+    },
+    {
+      "epoch": 35.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 315
+    },
+    {
+      "epoch": 35.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 316
+    },
+    {
+      "epoch": 35.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 317
+    },
+    {
+      "epoch": 35.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 318
+    },
+    {
+      "epoch": 35.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 319
+    },
+    {
+      "epoch": 35.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 320
+    },
+    {
+      "epoch": 35.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 321
+    },
+    {
+      "epoch": 35.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 322
+    },
+    {
+      "epoch": 35.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 323
+    },
+    {
+      "epoch": 36.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 324
+    },
+    {
+      "epoch": 36.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 325
+    },
+    {
+      "epoch": 36.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 326
+    },
+    {
+      "epoch": 36.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 327
+    },
+    {
+      "epoch": 36.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 328
+    },
+    {
+      "epoch": 36.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 329
+    },
+    {
+      "epoch": 36.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 330
+    },
+    {
+      "epoch": 36.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 331
+    },
+    {
+      "epoch": 36.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 332
+    },
+    {
+      "epoch": 37.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 333
+    },
+    {
+      "epoch": 37.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 334
+    },
+    {
+      "epoch": 37.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 335
+    },
+    {
+      "epoch": 37.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 336
+    },
+    {
+      "epoch": 37.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 337
+    },
+    {
+      "epoch": 37.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 338
+    },
+    {
+      "epoch": 37.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 339
+    },
+    {
+      "epoch": 37.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 340
+    },
+    {
+      "epoch": 37.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 341
+    },
+    {
+      "epoch": 38.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 342
+    },
+    {
+      "epoch": 38.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 343
+    },
+    {
+      "epoch": 38.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 344
+    },
+    {
+      "epoch": 38.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 345
+    },
+    {
+      "epoch": 38.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 346
+    },
+    {
+      "epoch": 38.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 347
+    },
+    {
+      "epoch": 38.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 348
+    },
+    {
+      "epoch": 38.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 349
+    },
+    {
+      "epoch": 38.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 350
+    },
+    {
+      "epoch": 39.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 351
+    },
+    {
+      "epoch": 39.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 352
+    },
+    {
+      "epoch": 39.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 353
+    },
+    {
+      "epoch": 39.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 354
+    },
+    {
+      "epoch": 39.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 355
+    },
+    {
+      "epoch": 39.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 356
+    },
+    {
+      "epoch": 39.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 357
+    },
+    {
+      "epoch": 39.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 358
+    },
+    {
+      "epoch": 39.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 359
+    },
+    {
+      "epoch": 40.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 360
+    },
+    {
+      "epoch": 40.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 361
+    },
+    {
+      "epoch": 40.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 362
+    },
+    {
+      "epoch": 40.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 363
+    },
+    {
+      "epoch": 40.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 364
+    },
+    {
+      "epoch": 40.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 365
+    },
+    {
+      "epoch": 40.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 366
+    },
+    {
+      "epoch": 40.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 367
+    },
+    {
+      "epoch": 40.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 368
+    },
+    {
+      "epoch": 41.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 369
+    },
+    {
+      "epoch": 41.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 370
+    },
+    {
+      "epoch": 41.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 371
+    },
+    {
+      "epoch": 41.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 372
+    },
+    {
+      "epoch": 41.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 373
+    },
+    {
+      "epoch": 41.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 374
+    },
+    {
+      "epoch": 41.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 375
+    },
+    {
+      "epoch": 41.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 376
+    },
+    {
+      "epoch": 41.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 377
+    },
+    {
+      "epoch": 42.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 378
+    },
+    {
+      "epoch": 42.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 379
+    },
+    {
+      "epoch": 42.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 380
+    },
+    {
+      "epoch": 42.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 381
+    },
+    {
+      "epoch": 42.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 382
+    },
+    {
+      "epoch": 42.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 383
+    },
+    {
+      "epoch": 42.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 384
+    },
+    {
+      "epoch": 42.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 385
+    },
+    {
+      "epoch": 42.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 386
+    },
+    {
+      "epoch": 43.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 387
+    },
+    {
+      "epoch": 43.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 388
+    },
+    {
+      "epoch": 43.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 389
+    },
+    {
+      "epoch": 43.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 390
+    },
+    {
+      "epoch": 43.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 391
+    },
+    {
+      "epoch": 43.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 392
+    },
+    {
+      "epoch": 43.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 393
+    },
+    {
+      "epoch": 43.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 394
+    },
+    {
+      "epoch": 43.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 395
+    },
+    {
+      "epoch": 44.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 396
+    },
+    {
+      "epoch": 44.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 397
+    },
+    {
+      "epoch": 44.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 398
+    },
+    {
+      "epoch": 44.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 399
+    },
+    {
+      "epoch": 44.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 400
+    },
+    {
+      "epoch": 44.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 401
+    },
+    {
+      "epoch": 44.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 402
+    },
+    {
+      "epoch": 44.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 403
+    },
+    {
+      "epoch": 44.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 404
+    },
+    {
+      "epoch": 45.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 405
+    },
+    {
+      "epoch": 45.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 406
+    },
+    {
+      "epoch": 45.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 407
+    },
+    {
+      "epoch": 45.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 408
+    },
+    {
+      "epoch": 45.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 409
+    },
+    {
+      "epoch": 45.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 410
+    },
+    {
+      "epoch": 45.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 411
+    },
+    {
+      "epoch": 45.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 412
+    },
+    {
+      "epoch": 45.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 413
+    },
+    {
+      "epoch": 46.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 414
+    },
+    {
+      "epoch": 46.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 415
+    },
+    {
+      "epoch": 46.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 416
+    },
+    {
+      "epoch": 46.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 417
+    },
+    {
+      "epoch": 46.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 418
+    },
+    {
+      "epoch": 46.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 419
+    },
+    {
+      "epoch": 46.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 420
+    },
+    {
+      "epoch": 46.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 421
+    },
+    {
+      "epoch": 46.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 422
+    },
+    {
+      "epoch": 47.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 423
+    },
+    {
+      "epoch": 47.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 424
+    },
+    {
+      "epoch": 47.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 425
+    },
+    {
+      "epoch": 47.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 426
+    },
+    {
+      "epoch": 47.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 427
+    },
+    {
+      "epoch": 47.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 428
+    },
+    {
+      "epoch": 47.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 429
+    },
+    {
+      "epoch": 47.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 430
+    },
+    {
+      "epoch": 47.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 431
+    },
+    {
+      "epoch": 48.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 432
+    },
+    {
+      "epoch": 48.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 433
+    },
+    {
+      "epoch": 48.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 434
+    },
+    {
+      "epoch": 48.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 435
+    },
+    {
+      "epoch": 48.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 436
+    },
+    {
+      "epoch": 48.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 437
+    },
+    {
+      "epoch": 48.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 438
+    },
+    {
+      "epoch": 48.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 439
+    },
+    {
+      "epoch": 48.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 440
+    },
+    {
+      "epoch": 49.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 441
+    },
+    {
+      "epoch": 49.11,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 442
+    },
+    {
+      "epoch": 49.22,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 443
+    },
+    {
+      "epoch": 49.33,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 444
+    },
+    {
+      "epoch": 49.44,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 445
+    },
+    {
+      "epoch": 49.56,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 446
+    },
+    {
+      "epoch": 49.67,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 447
+    },
+    {
+      "epoch": 49.78,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 448
+    },
+    {
+      "epoch": 49.89,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 449
+    },
+    {
+      "epoch": 50.0,
+      "learning_rate": 0.009382716049382716,
+      "loss": 0.0,
+      "step": 450
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 450,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 50,
+  "save_steps": 500,
+  "total_flos": 3.7170305826816e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-450/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf8250693281dc4d7b539c4eb47772eaa693f8d0810cebd330cac5a8dcfa98f7
+size 4728

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "HuggingFaceH4/zephyr-7b-beta",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 32768,
+  "model_type": "mistral",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": 2,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 10000.0,
+  "sliding_window": 4096,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.36.1",
+  "use_cache": true,
+  "vocab_size": 32000
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.36.1"
+}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5316c3c23e02e407a30dcc6db9e598c5379099b7caf5e33bbe839239dca565f0
+size 4943162240

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:34966c1a73ce3dc7501a6309f51d7d7556e599344ba987ab9dee2382e364232b
+size 4999819232

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6afc24cb3fb9a67c6d12f45f6d0523b076b4cabcfecb62a6e7b64f91bb813ad6
+size 4540516256

model.safetensors.index.json

@@ -0,0 +1,298 @@
+{
+  "metadata": {
+    "total_size": 14483464192
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00003-of-00003.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00003.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00003.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00003.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00003.safetensors",
+    "model.norm.weight": "model-00003-of-00003.safetensors"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,35 @@
+{
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

tokenizer_config.json

@@ -0,0 +1,48 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>"
+  ],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}\n{% if message['role'] == 'user' %}\n{{ '<|user|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'system' %}\n{{ '<|system|>\n' + message['content'] + eos_token }}\n{% elif message['role'] == 'assistant' %}\n{{ '<|assistant|>\n'  + message['content'] + eos_token }}\n{% endif %}\n{% if loop.last and add_generation_prompt %}\n{{ '<|assistant|>' }}\n{% endif %}\n{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "left",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": true
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf8250693281dc4d7b539c4eb47772eaa693f8d0810cebd330cac5a8dcfa98f7
+size 4728

training_params.json

@@ -0,0 +1,46 @@
+{
+    "model": "HuggingFaceH4/zephyr-7b-beta",
+    "project_name": "/tmp/model",
+    "data_path": "pratikthakkar007/autotrain-data-autotrain-wuiwp-o6gob",
+    "train_split": "train",
+    "valid_split": null,
+    "add_eos_token": true,
+    "block_size": 1024,
+    "model_max_length": 2048,
+    "trainer": "default",
+    "use_flash_attention_2": false,
+    "log": "none",
+    "disable_gradient_checkpointing": false,
+    "logging_steps": -1,
+    "evaluation_strategy": "epoch",
+    "save_total_limit": 1,
+    "save_strategy": "epoch",
+    "auto_find_batch_size": true,
+    "mixed_precision": "fp16",
+    "lr": 0.01,
+    "epochs": 50,
+    "batch_size": 2,
+    "warmup_ratio": 0.1,
+    "gradient_accumulation": 1,
+    "optimizer": "adamw_torch",
+    "scheduler": "linear",
+    "weight_decay": 0.0,
+    "max_grad_norm": 1.0,
+    "seed": 42,
+    "apply_chat_template": false,
+    "quantization": "int4",
+    "target_modules": "",
+    "merge_adapter": true,
+    "peft": true,
+    "lora_r": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "model_ref": null,
+    "dpo_beta": 0.1,
+    "prompt_text_column": "autotrain_prompt",
+    "text_column": "autotrain_text",
+    "rejected_text_column": "autotrain_rejected_text",
+    "push_to_hub": true,
+    "repo_id": "pratikthakkar007/autotrain-wuiwp-o6gob",
+    "username": "pratikthakkar007"
+}