Error when trying to use vLLM as the backend

by BiboyQG - opened Jul 20

Jul 20

AttributeError: 'MergedColumnParallelLinear' object has no attribute 'weight'. Did you mean: 'qweight'?

Jul 20

I assume this is an vLLM issue, so will close now.

BiboyQG changed discussion status to closed Jul 20

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment