eos token seems not working
maybe there's a quick fix? I'm digging.
I have discovered a fix that requires retraining the model. In progress.
This model still completely blows me away with its intelligence.
How do you mean EOS token does not work?
It just keeps talking and doesn't stop
I have discovered a fix that requires retraining the model. In progress.
This model still completely blows me away with its intelligence.
Looking forward to it! Though I will have to wait for a TheBloke quantization.
Congrats on the model!
There is a workaround:
In your system message, tell the model to write <|FINISHED|> when it is finished
Then have your client stop generating when it sees that string.
Argh, that's the known problem :) Yeah, adding <|FINISHED|> might work as a temporary workaround. But I'd prefer to see this model (trained on StellarBright again) with correct EOS handling too
Yeah I decided to go ahead and use StellarBright since the OpenAI stuff only comes out when you deviate from ChatML
I am training it again but - suddenly Azure is WAY more popular so my spot instance keeps getting evicted. It's gotten much worse in the last week.
dolphin-2.2-70b will be finished in about a week.