phi-2-ele

README.md

@@ -35,7 +35,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0001
+- learning_rate: 0.1
 - train_batch_size: 1
 - eval_batch_size: 8
 - seed: 42
@@ -43,7 +43,7 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 2
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 20
+- num_epochs: 5
 
 ### Training results
 
@@ -52,7 +52,7 @@ The following hyperparameters were used during training:
 ### Framework versions
 
 - PEFT 0.11.1
-- Transformers 4.41.2
+- Transformers 4.42.3
 - Pytorch 2.3.0+cu121
 - Datasets 2.20.0
 - Tokenizers 0.19.1

adapter_config.json

@@ -20,12 +20,12 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
     "q_proj",
-    "v_proj",
+    "k_proj",
+    "fc2",
     "dense",
     "fc1",
-    "fc2"
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fde1884af72a363500e2cede347dde01beaaf37ec7f8482d04697b6f7a04d328
+oid sha256:e4914a5bfa292350e2157b1b7acb19f111fe7112f9db36ab82ff14b0024d1cee
 size 94422368

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4b29550fb483ff61e7d289023e7fa4ccb29ba8b11852c6ddb9f7dc3a80175a30
+oid sha256:9c847c4f0c40e5d0fc558806eec4490d096d7529cb8c0da3a2272728e4dcb39d
 size 5368