"Supports a context length of 160k through yarn settings."
#1
by
mclassHF2023
- opened
How would this be achieved precisely? I thought with GGUF files there is no config.json anymore?
Supports a context length of 160k through yarn settings.
Btw: I would be happy with about 16k context already, it doesn't have to be 160k context.
I think you should just need to set the RoPE Frequency Scale when loading the model in LM Studio (if you hold alt while loading it you should be able to set a higher context length and use the RoPE frequency scale for better scaling)
if you want 16k context I think you want a frequency scale of 4