Update README.md
Browse files
README.md
CHANGED
@@ -209,6 +209,8 @@ I chose the weird fiction example to see if the LLM understood a niche genre. It
|
|
209 |
|
210 |
This was trained with an r 128 qlora over 2 epochs on a mix of public & private datasets using Axolotl.
|
211 |
|
|
|
|
|
212 |
This took approx. 51 hours on 1x NVIDIA A100 80GB GPU.
|
213 |
|
214 |
[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
|
|
|
209 |
|
210 |
This was trained with an r 128 qlora over 2 epochs on a mix of public & private datasets using Axolotl.
|
211 |
|
212 |
+
Training was performed with a 16384 seq len to try to preserve Llama 3.1's long context.
|
213 |
+
|
214 |
This took approx. 51 hours on 1x NVIDIA A100 80GB GPU.
|
215 |
|
216 |
[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
|