922CA commited on
Commit
30390ad
1 Parent(s): 787df88

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -65,11 +65,11 @@ Example:
65
 
66
  # llama-2-13b-tagalog-v0.3 loras (09/01-02/2023)
67
  * Fine tuned on experimental datasets of ~1k items (Tagalog-focused dataset, based off Tagalog sentences augmented by LLaMA-2-13b base to create a 3-turn dialogue dataset between Human and Assistant)
68
- * 3 fine-tuned for 1 epoch, rank = 16
69
- * 3a for 1 epoch, rank = 8
70
  * 3b for 2 epochs
71
  * 3c for 1 epoch, lr = 1e-4, warmup steps = 0.1
72
- * 3d for 1 epoch, lr = 2e-4, warmup steps = 0.1, rank = 32, lora alpha = 64
73
- * 3e for 2 epochs, lr = 2e-4, warmup steps = 0.1, rank = 32, lora alpha = 64
74
  * From LLaMA-2-13b
75
  * Trying LLaMA-2-13b chat/other base and curated dataset for next attempts
 
65
 
66
  # llama-2-13b-tagalog-v0.3 loras (09/01-02/2023)
67
  * Fine tuned on experimental datasets of ~1k items (Tagalog-focused dataset, based off Tagalog sentences augmented by LLaMA-2-13b base to create a 3-turn dialogue dataset between Human and Assistant)
68
+ * 3 fine-tuned for 1 epoch, rank = 16, lora alpha = 32
69
+ * 3a with rank = 8
70
  * 3b for 2 epochs
71
  * 3c for 1 epoch, lr = 1e-4, warmup steps = 0.1
72
+ * 3d with lr = 2e-4, rank = 32, lora alpha = 64
73
+ * 3e for 2 epochs
74
  * From LLaMA-2-13b
75
  * Trying LLaMA-2-13b chat/other base and curated dataset for next attempts