Thanks for this and would *love* to see an int4 version

#3
by CyberTimon - opened

Hello @JosephusCheung

I really like this model but I really can't quantize this to int4, but it should be possible as this (https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-int4) is working.

Thank you very much!

Sign up or log in to comment