GGML Quant?

#1
by evangineer - opened

Hi, would it be possible to produce a quantised GGML version for local use, please.

I'm really interested in trying Retrieval Augmented Generation using this model to chat with newly published scientific articles.

Sign up or log in to comment