LLM-Alchemy-Chamber/mistral-instruct-generation

README.md

@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.8556
+- Loss: 0.8576
 
 ## Model description
 
@@ -48,16 +48,16 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.1522        | 0.03  | 10   | 1.1246          |
-| 1.0849        | 0.07  | 20   | 1.0571          |
-| 1.0144        | 0.1   | 30   | 0.9943          |
-| 0.9654        | 0.13  | 40   | 0.9525          |
-| 0.9383        | 0.16  | 50   | 0.9212          |
-| 0.8954        | 0.2   | 60   | 0.8979          |
-| 0.8671        | 0.23  | 70   | 0.8798          |
-| 0.9088        | 0.26  | 80   | 0.8664          |
-| 0.8696        | 0.3   | 90   | 0.8584          |
-| 0.8281        | 0.33  | 100  | 0.8556          |
+| 1.1699        | 0.03  | 10   | 1.1218          |
+| 1.0878        | 0.07  | 20   | 1.0544          |
+| 1.0525        | 0.1   | 30   | 0.9935          |
+| 0.9611        | 0.13  | 40   | 0.9529          |
+| 0.931         | 0.16  | 50   | 0.9230          |
+| 0.9212        | 0.2   | 60   | 0.8993          |
+| 0.8918        | 0.23  | 70   | 0.8817          |
+| 0.8808        | 0.26  | 80   | 0.8683          |
+| 0.8575        | 0.3   | 90   | 0.8604          |
+| 0.8848        | 0.33  | 100  | 0.8576          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,14 +20,14 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
-    "v_proj",
-    "down_proj",
+    "up_proj",
     "o_proj",
-    "k_proj",
+    "down_proj",
+    "q_proj",
     "gate_proj",
     "lm_head",
-    "up_proj"
+    "k_proj",
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ef037664ff1f241c9f05dbc8529a74c4b3f73f84043e90d3ed3dab3fdf3797f5
+oid sha256:2144d3d7ba49f9a6d25161173a1ced5aa06c73567931c64e8c7e06b9052e3cb9
 size 751667752

runs/Mar17_02-55-22_llm-back-project-workbench-0/events.out.tfevents.1710644123.llm-back-project-workbench-0.376.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aebc90b5ee6e9f6903e482d129d0f7b4838c56c879fa407cba00d8bd4f19e271
+size 9431

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6bbec64aaf2fa220f49f1f4d5180dd6a7e764015a902bf93f9475b73fcce36fa
+oid sha256:8a1636e27ff461eb334da7d10b79596f6474584607f17ad70a1ebbafb1d09dc0
 size 4283