Update README.md

README.md

@@ -142,4 +142,50 @@ answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
 from peft import LoraConfig, PeftModelForQuestionAnswering
 from transformers import AutoModelForQuestionAnswering, AutoTokenizer
 model_name = "sjrhuschlee/deberta-v3-large-squad2"
-```
+```
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 24
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 1
+- total_train_batch_size: 24
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 4.0
+
+### LoRA Config
+```
+{
+  "base_model_name_or_path": "microsoft/deberta-v3-large",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "lora_alpha": 32,
+  "lora_dropout": 0.1,
+  "modules_to_save": ["qa_outputs"],
+  "peft_type": "LORA",
+  "r": 8,
+  "target_modules": [
+    "query_proj",
+    "key_proj",
+    "value_proj",
+    "dense"
+  ],
+  "task_type": "QUESTION_ANS"
+}
+```
+
+### Framework versions
+
+- Transformers 4.30.0.dev0
+- Pytorch 2.0.1+cu117
+- Datasets 2.12.0
+- Tokenizers 0.13.3