What is the context length of this model?
#4
by
MrHillsss
- opened
I thought it was 32k but I'm getting weird results above 4k. I get the same issue with Mixtral exl2, but above 6k context. Maybe it's an issue with MoEs?
It's 4k can be stretched more with RoPE, it's the same context than Llama2 model, I think the last mergekit update corrected that in the config it output now
How do you stretch it with RoPE? I'm having issues setting the RoPE settings to effectively stretch this... What settings do you use?
I'm seeing the same behavior as MrHillsss. Impressive RP performance within 4k though, characters won't drop their pants immediately if it doesn't make sense with their card.