Reccomended params?
Hello @lewtun , What are the recommended default params for coding use? (specifically generating code, maybe some tool use)
I think it would help people get the best results, i did check independently to work out what the recommended ones were, but couldn't find anything concrete specifically to this model, although the base model has recommendations.
I am mostly interested in, temperature, top_p & top_k but any other recommended changed params from defaults would be useful.
See below for current supported params in ollama, and their default values:
https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter
Note: In this example i am using ollama, but it should apply to most inference software that uses standard engines like llama.cpp.
Also to clarify would this advice, be the same for all sizes of this model? (like 7B) (assuming the same training process for both, haven't read the code fully yet)
Many thanks,
James Clarke
https://huggingface.co/open-r1/OlympicCoder-32B/blob/main/generation_config.json
And as it is specifically a coder model, I guess those are the recommended params for code :D
So in your modelfile, you can just copy paste this ;)
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER repeat_penalty 1.1
PARAMETER top_k 20
https://huggingface.co/open-r1/OlympicCoder-32B/blob/main/generation_config.json
And as it is specifically a coder model, I guess those are the recommended params for code :DSo in your modelfile, you can just copy paste this ;)
PARAMETER temperature 0.7 PARAMETER top_p 0.8 PARAMETER repeat_penalty 1.1 PARAMETER top_k 20
Ah, thanks @owao , I should have known to check there for params, my apologies.
Will close issue as solved.
@Impulse2000 Nah don't have to present apologies, we're all learning here! And could have been me few months ago ;)