End of training

README.md

@@ -11,7 +11,38 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
+[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)## axolotl config
+```yaml
+base_model: openaccess-ai-collective/tiny-mistral
+flash_attention: true
+sequence_len: 1024
+load_in_8bit: true
+adapter: lora
+lora_r: 32
+lora_alpha: 64
+lora_dropout: 0.05
+lora_target_linear: true
+val_set_size: 0.1
+special_tokens:
+  unk_token: <unk>
+  bos_token: <s>
+  eos_token: </s>
+datasets:
+  - path: mhenrichsen/alpaca_2k_test
+    type: alpaca
+num_epochs: 2
+micro_batch_size: 2
+gradient_accumulation_steps: 1
+output_dir: temp_dir
+learning_rate: 0.00001
+optimizer: adamw_torch
+lr_scheduler: cosine
+max_steps: 20
+save_steps: 10
+eval_steps: 10
+hub_model_id: hamel/axolotl-test
+dataset_processes: 1
+```
 # axolotl-test
 
 This model is a fine-tuned version of [openaccess-ai-collective/tiny-mistral](https://huggingface.co/openaccess-ai-collective/tiny-mistral) on the None dataset.

adapter_config.json

@@ -16,13 +16,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "gate_proj",
     "up_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj",
     "down_proj",
     "v_proj",
-    "o_proj",
-    "k_proj",
-    "q_proj"
+    "k_proj"
   ],
   "task_type": "CAUSAL_LM"
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b1021d6333395951d27fb257dbbaf54629e4bc56bb7c684a2ac1f6f336d3ee80
+oid sha256:bb0d0bf616b0ec3bd6fd2a39c4d00678d2582666fedfaa5a5ba722e303e5df34
 size 49035696

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4303f89612d33a141de3e34c1f741780fdec24e3dd717b397e5cb97683bde0e5
+oid sha256:c05a806adf74045eb5ac3ed69e66cc42c10590328fa2414dd57e9f02453b12d8
 size 5176