Q6_k vram requirements
#1
by
mjh657
- opened
How much vram is needed to run this at Q6_k?
That depends majorly on your setting (context length, attention, k-v cache) and even your software.
You cna use spaces such as https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator to get a rough estimate.
mradermacher
changed discussion status to
closed