Update README.md
Browse files
README.md
CHANGED
@@ -65,11 +65,11 @@ Example:
|
|
65 |
|
66 |
# llama-2-13b-tagalog-v0.3 loras (09/01-02/2023)
|
67 |
* Fine tuned on experimental datasets of ~1k items (Tagalog-focused dataset, based off Tagalog sentences augmented by LLaMA-2-13b base to create a 3-turn dialogue dataset between Human and Assistant)
|
68 |
-
* 3 fine-tuned for 1 epoch, rank = 16
|
69 |
-
* 3a
|
70 |
* 3b for 2 epochs
|
71 |
* 3c for 1 epoch, lr = 1e-4, warmup steps = 0.1
|
72 |
-
* 3d
|
73 |
-
* 3e for 2 epochs
|
74 |
* From LLaMA-2-13b
|
75 |
* Trying LLaMA-2-13b chat/other base and curated dataset for next attempts
|
|
|
65 |
|
66 |
# llama-2-13b-tagalog-v0.3 loras (09/01-02/2023)
|
67 |
* Fine tuned on experimental datasets of ~1k items (Tagalog-focused dataset, based off Tagalog sentences augmented by LLaMA-2-13b base to create a 3-turn dialogue dataset between Human and Assistant)
|
68 |
+
* 3 fine-tuned for 1 epoch, rank = 16, lora alpha = 32
|
69 |
+
* 3a with rank = 8
|
70 |
* 3b for 2 epochs
|
71 |
* 3c for 1 epoch, lr = 1e-4, warmup steps = 0.1
|
72 |
+
* 3d with lr = 2e-4, rank = 32, lora alpha = 64
|
73 |
+
* 3e for 2 epochs
|
74 |
* From LLaMA-2-13b
|
75 |
* Trying LLaMA-2-13b chat/other base and curated dataset for next attempts
|