Llama-3.2 Quantization - a neuralmagic Collection

neuralmagic 's Collections

Sparse-Llama-3.1-2of4

Vision Language Models Quantization

FP8 LLMs for vLLM

Llama-3.2 Quantization

Llama-3.1 Quantization

INT8 LLMs for vLLM

INT4 LLMs for vLLM

Sparse Foundational Llama 2 Models

Compression Papers

DeepSparse Sparse LLMs

Sparse Finetuning MPT

Compressed LLMs from the Community

Llama-3.2 Quantization

updated Sep 26

Llama 3.2 models quantized by Neural Magic

neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 144k • 14
neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2 • 24.3k • 5
neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 1.14k • 2
neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9 • 2.11k • 2
neuralmagic/Llama-3.2-1B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16 • 3.7k • 4
neuralmagic/Llama-3.2-3B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16 • 7.63k • 1
neuralmagic/Llama-3.2-1B-Instruct-FP8

Text Generation • Updated Oct 16 • 262k • 1
neuralmagic/Llama-3.2-3B-Instruct-FP8

Text Generation • Updated Oct 16 • 15.9k • 2
neuralmagic/Llama-3.2-1B-FP8

Updated Oct 9 • 202