Hugging Face's logo

nm-testing
/
DeepSeek-R1-Distill-Llama-70B-FP8-dynamic

Text Generation
Transformers
Safetensors
English
llama
deepseek
fp8
vllm
conversational
text-generation-inference
Inference Endpoints
compressed-tensors
Model card Files Community
1

Unsupported Scale Group Shapes in CUTLASS 3.x GEMM Kernels When Using vLLM OpenAI Docker Image v0.7.1

#1
by NikolaSigmoid - opened
Discussion
NikolaSigmoid

I used vllm and got this bug

Sign up or log in to comment