Llama 3.1 GPTQ, AWQ, and BNB Quants - a hugging-quants Collection

hugging-quants 's Collections

Gemma2 AWQ Quants

Llama 3.2 3B & 1B GGUF Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

updated Sep 26

Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • Updated Sep 13 • 901 • 37
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4

Text Generation • Updated Sep 16 • 1.09k • 5
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 563 • 16
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 179k • 92
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit

Text Generation • Updated Nov 22 • 21.5k • 27
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 101k • 22
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 464k • 61
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4

Text Generation • Updated Aug 8 • 4.3k • 7
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 94.9k • 21