Hari7696/peft-ia3

README.md

@@ -33,13 +33,14 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.001
+- learning_rate: 0.003
 - train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 2
+- lr_scheduler_warmup_steps: 50
+- num_epochs: 4
 
 ### Training results
 
@@ -50,5 +51,5 @@ The following hyperparameters were used during training:
 - PEFT 0.12.0
 - Transformers 4.42.4
 - Pytorch 2.3.1+cu121
-- Datasets 2.20.0
+- Datasets 2.21.0
 - Tokenizers 0.19.1

adapter_config.json

@@ -11,8 +11,10 @@
   "peft_type": "IA3",
   "revision": null,
   "target_modules": [
+    "k",
+    "q",
     "v",
-    "q"
+    "o"
   ],
   "task_type": "SEQ_2_SEQ_LM"
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d1c9dc6003639d7d38984860d36c7a9c38dcac88cc070132a2442b742af65587
-size 230432
+oid sha256:8f2a78c02265fbfaeac6c5725eeaade5ee667aa6e5aabab19ef05308ea7ac247
+size 460888

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:420a0644455b70b18e0d63169d82689c4097655ffdc0c3701ba1815de8abfd8f
+oid sha256:c286501d83149fd15352a50b349d8b9d04bc2d4075f51ac7b8259928373311da
 size 5048