TheBloke
/

Mixtral-8x7B-Instruct-v0.1-AWQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Resources

View closed (0)

Use ybelkada/Mixtral-8x7B-Instruct-v0.1-AWQ with VLLM instead

#10 opened 7 months ago by

Inference taking too much time

#9 opened 9 months ago by

Update README.md

#8 opened 9 months ago by

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

#7 opened 9 months ago by

TGI - response is an empty string

#6 opened 10 months ago by

OC is not a multiple of cta_N = 64

#5 opened 10 months ago by

lazyDataScientist

Not supporting with TGI

#4 opened 10 months ago by

abhishek3jangid

always getting 0 in output

#3 opened 11 months ago by

OOM under vLLM even with 80GB GPU

#2 opened 11 months ago by

Not supported for TGI > 1.3 ?

#1 opened 11 months ago by