LLM-Alchemy-Chamber/mistral-instruct-gen-v4

README.md

@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.8576
+- Loss: 0.7630
 
 ## Model description
 
@@ -36,7 +36,7 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 2.5e-05
-- train_batch_size: 16
+- train_batch_size: 10
 - eval_batch_size: 8
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
@@ -48,16 +48,16 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.1699        | 0.03  | 10   | 1.1218          |
-| 1.0878        | 0.07  | 20   | 1.0544          |
-| 1.0525        | 0.1   | 30   | 0.9935          |
-| 0.9611        | 0.13  | 40   | 0.9529          |
-| 0.931         | 0.16  | 50   | 0.9230          |
-| 0.9212        | 0.2   | 60   | 0.8993          |
-| 0.8918        | 0.23  | 70   | 0.8817          |
-| 0.8808        | 0.26  | 80   | 0.8683          |
-| 0.8575        | 0.3   | 90   | 0.8604          |
-| 0.8848        | 0.33  | 100  | 0.8576          |
+| 0.9965        | 0.01  | 10   | 0.9608          |
+| 0.9611        | 0.02  | 20   | 0.9045          |
+| 0.8601        | 0.02  | 30   | 0.8574          |
+| 0.8382        | 0.03  | 40   | 0.8280          |
+| 0.8326        | 0.04  | 50   | 0.8072          |
+| 0.7815        | 0.05  | 60   | 0.7904          |
+| 0.796         | 0.06  | 70   | 0.7786          |
+| 0.7668        | 0.07  | 80   | 0.7701          |
+| 0.7774        | 0.07  | 90   | 0.7648          |
+| 0.7699        | 0.08  | 100  | 0.7630          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,14 +20,14 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "up_proj",
+    "q_proj",
     "o_proj",
     "down_proj",
-    "q_proj",
-    "gate_proj",
     "lm_head",
-    "k_proj",
-    "v_proj"
+    "v_proj",
+    "gate_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2144d3d7ba49f9a6d25161173a1ced5aa06c73567931c64e8c7e06b9052e3cb9
+oid sha256:e6d7b3b0957ac0b7bf695d53bc989c422c608927ccd72d92de5ef0f8a27b7bb5
 size 751667752

runs/Mar19_08-05-38_llm-genenerative-workbench-0/events.out.tfevents.1710835539.llm-genenerative-workbench-0.748.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc977a7b52f09871e5423ae275ff6e0b777d8fe89bfd9dc021b27d56cfd8094f
+size 4883

runs/Mar19_12-21-34_llm-genenerative-workbench-0/events.out.tfevents.1710850894.llm-genenerative-workbench-0.629.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5d75413fded0b3afb1535a7b540c73b1a741e57c7a04cdc40fba454bf535c0f
+size 9431

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8a1636e27ff461eb334da7d10b79596f6474584607f17ad70a1ebbafb1d09dc0
+oid sha256:c954fb7eb42f3f555738afc9de64de8c969c8194d16790ace3d1f155aae492b9
 size 4283