Determinism
#3
by
s-vh
- opened
Hey, I'd like to ask if the model is deterministic?
Thanks!
Should be for the most part if you decode greedily i.e. you should get the generation commented out here https://huggingface.co/allenai/OLMoE-1B-7B-0924#use for that prompt written there
And if I want to make sure it is fully deterministic can I use manual seed in my code?
Keep in mind that GPUs are not completely deterministic, just barely enough for practical purposes.
dirkgr
changed discussion status to
closed