Reccomended params?

#5
by Impulse2000 - opened

Hello @lewtun , What are the recommended default params for coding use? (specifically generating code, maybe some tool use)

I think it would help people get the best results, i did check independently to work out what the recommended ones were, but couldn't find anything concrete specifically to this model, although the base model has recommendations.

I am mostly interested in, temperature, top_p & top_k but any other recommended changed params from defaults would be useful.

See below for current supported params in ollama, and their default values:
https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter

Note: In this example i am using ollama, but it should apply to most inference software that uses standard engines like llama.cpp.

Also to clarify would this advice, be the same for all sizes of this model? (like 7B) (assuming the same training process for both, haven't read the code fully yet)

Many thanks,
James Clarke

https://huggingface.co/open-r1/OlympicCoder-32B/blob/main/generation_config.json
And as it is specifically a coder model, I guess those are the recommended params for code :D

So in your modelfile, you can just copy paste this ;)

PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER repeat_penalty 1.1
PARAMETER top_k 20

https://huggingface.co/open-r1/OlympicCoder-32B/blob/main/generation_config.json
And as it is specifically a coder model, I guess those are the recommended params for code :D

So in your modelfile, you can just copy paste this ;)

PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER repeat_penalty 1.1
PARAMETER top_k 20

Ah, thanks @owao , I should have known to check there for params, my apologies.

Will close issue as solved.

Impulse2000 changed discussion status to closed

@Impulse2000 Nah don't have to present apologies, we're all learning here! And could have been me few months ago ;)

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment