Could you make a smaller version?
#1
by
BoscoTheDog
- opened
I don't know if this is a wild question, but is it possible to create a smaller version of the model that could, for example, run on a mobile phone?
Or are there perhaps smaller models you'd recommend for this purpose?
Interesting, I don't see any Phi2 variant in HF. I don't mind doing that but are you sure you can't get a heavily quantized version of this model to run on your phone? This model has a Q2_K GGUF and that might be worth trying still.
I will soon :) https://huggingface.co/microsoft/Phi-3-mini-128k-instruct
Fantastic :-)
Any progress?
@victunes I hope you'll still pursue this, it would be wonderful to have a smaller model that can run in the browser.
BoscoTheDog
changed discussion status to
closed
BoscoTheDog
changed discussion status to
open