Update README.md

Files changed (1) hide show

README.md CHANGED Viewed

@@ -53,7 +53,7 @@ I'll try resolving it with a light merge ASAP, it seems like the wrong weight is
 I appreciate all feedback on any of my models, you can use:
 * [My Discord server](https://discord.gg/AJwZuu7Ncx) - requires Discord.
-* [The Community tab](https://huggingface.co/invisietch/MiS-Firefly-v0.1-22B/discussions) - requires HF login.
 * Discord DMs to **invisietch**.
 Your feedback is how I improve these models for future versions.
@@ -95,6 +95,8 @@ The first stage of my training was a single epoch at low LR over a 474 million t
 I followed this up with a coherence, decensorship & roleplay finetune over a 172 million token instruct dataset over two epochs.
 Total training time was about 32hrs on 4x Nvidia A100 80GB.
 <img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>

 I appreciate all feedback on any of my models, you can use:
 * [My Discord server](https://discord.gg/AJwZuu7Ncx) - requires Discord.
+* [The Community tab](https://huggingface.co/invisietch/MiS-Firefly-v0.2-22B/discussions) - requires HF login.
 * Discord DMs to **invisietch**.
 Your feedback is how I improve these models for future versions.
 I followed this up with a coherence, decensorship & roleplay finetune over a 172 million token instruct dataset over two epochs.
+I did a slerp merge of epoch 1 into epoch 2 at a light weight which resolved the name-spelling issues on quantized versions of Firefly v0.1.
 Total training time was about 32hrs on 4x Nvidia A100 80GB.
 <img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>