invisietch commited on
Commit
829008d
·
verified ·
1 Parent(s): 8f6266f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -209,6 +209,8 @@ I chose the weird fiction example to see if the LLM understood a niche genre. It
209
 
210
  This was trained with an r 128 qlora over 2 epochs on a mix of public & private datasets using Axolotl.
211
 
 
 
212
  This took approx. 51 hours on 1x NVIDIA A100 80GB GPU.
213
 
214
  [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
 
209
 
210
  This was trained with an r 128 qlora over 2 epochs on a mix of public & private datasets using Axolotl.
211
 
212
+ Training was performed with a 16384 seq len to try to preserve Llama 3.1's long context.
213
+
214
  This took approx. 51 hours on 1x NVIDIA A100 80GB GPU.
215
 
216
  [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)