ArunKr
/

LLM-Models

Inference Endpoints

Model card Files Files and versions Community

LLM-Models / quantize.sh

Arun Kumar Tiwary

Upload folder using huggingface_hub

ae919d8 verified 6 months ago

103 Bytes

	./quantize ./Meta-Llama-3-8B-Instruct_fp16.bin output/Meta-Llama-3-8B-Instruct_fp16_Q4_K_M.bin Q4_K_M