Crystalcareai commited on
Commit
b7cba62
·
verified ·
1 Parent(s): a22e5a4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -26,7 +26,7 @@ Discord: https://discord.gg/8fbBeC7ZGx
26
 
27
  <img src="https://cdn-uploads.huggingface.co/production/uploads/63111b2d88942700629f5771/ldkN1J0WIDQwU4vutGYiD.png" width="600" />
28
 
29
- We have retrained our LLama-3-8b fine tune to address behavioral issues in our initial 2.9 dataset. Specifically, Systemchat was causing the model to be *too* reliant on the system prompt. Additionally, it had an occasional quirk that would cause the model to overly reference the system prompt. We also found generation length was at times not sufficient for any given task. We identified the culprit as Ultrachat. Accounting for these concerns, we removed systemchat and ultrachat from the dataset. It is otherwise identical to dolphin-2.9.
30
 
31
  Our appreciation for the sponsors of Dolphin 2.9.1:
32
  - [Crusoe Cloud](https://crusoe.ai/) - provided excellent on-demand 8xL40S node
 
26
 
27
  <img src="https://cdn-uploads.huggingface.co/production/uploads/63111b2d88942700629f5771/ldkN1J0WIDQwU4vutGYiD.png" width="600" />
28
 
29
+ We have retrained our LLama-3-8b fine tune to address behavioral issues in the initial 2.9 dataset. Specifically, Systemchat was causing the model to be *too* reliant on the system prompt. Additionally, it had an occasional quirk that would cause the model to overly reference the system prompt. We also found generation length was at times not sufficient for any given task. We identified the culprit as Ultrachat. Accounting for these concerns, we removed systemchat and ultrachat from the dataset. It is otherwise identical to dolphin-2.9.
30
 
31
  Our appreciation for the sponsors of Dolphin 2.9.1:
32
  - [Crusoe Cloud](https://crusoe.ai/) - provided excellent on-demand 8xL40S node