Model speed
#1
by
mduran159
- opened
the meta-llama/Llama-3.2-1B-Instruct is even faster than this model. Weird thing, because last version for 3.1 was great. I will be waiting for the fix, for sure.
mduran159
changed discussion status to
closed
nevermind Im an Idiot xd
mduran159
changed discussion status to
open
Well, now comparing, Its almost at the same speed than a Llama-3.2-3B-Instruct. A model with 2B more of parameters and this model is supossed to be faster. Maybe I'm still being an idiot and I'm missing something, but I think now models are in equal conditions and unsloth/Llama-3.2-1B-Instruct-bnb-4bit is almost the same speed than a meta-llama/Llama-3.2-3B-Instruct.
Maybe I'm missing something else to make this model work properly
mduran159
changed discussion status to
closed
mduran159
changed discussion title from
This stuff is so slow, even slower than original model.
to I think this model is slower than original model.
What are you using this model on?
shimmyshimmer
changed discussion title from
I think this model is slower than original model.
to Model speed