Could you make a smaller version?

#1
by BoscoTheDog - opened

I don't know if this is a wild question, but is it possible to create a smaller version of the model that could, for example, run on a mobile phone?

Or are there perhaps smaller models you'd recommend for this purpose?

Interesting, I don't see any Phi2 variant in HF. I don't mind doing that but are you sure you can't get a heavily quantized version of this model to run on your phone? This model has a Q2_K GGUF and that might be worth trying still.

A Phi2 version would be great! If you make it, I will make it part of my current project, which should make it useful for a lot of people.

sneak_preview.png
It's a 100% browser-based, local way to run smaller AI's, all designed/parameterized for specific tasks.

Fantastic :-)

Any progress?

@victunes I hope you'll still pursue this, it would be wonderful to have a smaller model that can run in the browser.

BoscoTheDog changed discussion status to closed
BoscoTheDog changed discussion status to open

Sign up or log in to comment