4bit
/

WizardLM-13B-Uncensored-4bit-128g

Text Generation

Transformers

llama

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

WizardLM-13B-Uncensored-4bit-128g / README.md

camenduru

thanks to ausboss ❤

81d47ba over 1 year ago

preview code

raw

history blame

No virus

242 Bytes

quantized this model

CUDA_VISIBLE_DEVICES=0 python llama.py ehartford/WizardLM-13B-Uncensored c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors 4bit-128g.safetensors