LoneStriker
/

deepseek-llm-67b-chat-5.0bpw-h6-exl2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Can we get a 6bpw for 48gb gpus?

#1

by AnonZ123 - opened Nov 29, 2023

Nov 29, 2023

I've seen you commonly stop at 5bpw, I'd love to see a 6bpw for this model.

Nov 29, 2023

Thank you!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment