TheBloke

/

Vicuna-13B-1.1-GPTQ

like 208

@laek https://python.langchain.com/en/latest/modules/models/llms/integrations/llamacpp.html

That link is for llama.cpp which couldn't be used with this GPTQ model and GPU inference, but could be used with the Vicuna GGMLs that you can download

For GPTQ GPU inference with Langchain, check out https://github.com/cxfcxf/embeddings