llama.cpp ggml 4-bit quantized please

by Sam2x - opened Apr 7, 2023

Apr 7, 2023

Hi,

First thank you for the effort to do this model , i think it will be great to generate a quantized version to test it on cpu!

Best,

Owner Apr 17, 2023

Thanks for your interest!

Due to the license of the Meta LLaMA model, the quantized merged model cannot be shared.

I would recommend reading this section for how to quantize it by yourself.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment