chtan
/

gpt4-alpaca-lora_mlp-65b

Text2Text Generation

Model card Files Files and versions Community

chtan commited on May 5, 2023

Commit

29c4f2e

•

1 Parent(s): 3d067d8

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -19,11 +19,11 @@ torchrun --nproc_per_node=8 finetune.py \
     --output_dir './gpt4-alpaca-lora_mlp-65b' \
     --batch_size 128 \
     --micro_batch_size 2 \
-    --num_epochs 3 \
     --learning_rate 1e-4 \
     --cutoff_len 512 \
     --val_set_size 2000 \
-    --lora_r 8 \
     --lora_alpha 16 \
     --lora_dropout 0.05 \
     --lora_target_modules '[gate_proj,down_proj,up_proj]' \

     --output_dir './gpt4-alpaca-lora_mlp-65b' \
     --batch_size 128 \
     --micro_batch_size 2 \
+    --num_epochs 10 \
     --learning_rate 1e-4 \
     --cutoff_len 512 \
     --val_set_size 2000 \
+    --lora_r 16 \
     --lora_alpha 16 \
     --lora_dropout 0.05 \
     --lora_target_modules '[gate_proj,down_proj,up_proj]' \