More qunt models

by mtasic85 - opened Sep 15

Sep 15

Can you upload q4_k_m and q8_0 models?

Pints.ai org Sep 16

@mtasic85 at that level of quantisation, they will practically be not very useful though.

Maybe you can let me know why would you want to consider those?

Sep 16

I quantized both versions locally and did not have much trouble.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment