Error with Text-Embeddings-Inference Container using JINA-v3 Model
#1
by
ilhamdprastyo
- opened
I'm encountering an error while running text-embeddings-inference with JINA-v3 model on a single NVIDIA A6000 GPU. Here's my setup and error:
Docker-compose:
version: '3.8'
services:
text-embeddings-inference-jina-v3:
image: ghcr.io/huggingface/text-embeddings-inference:latest
container_name: text_embeddings_inference_jina_v3
ports:
- "8083:80"
restart: always
volumes:
- /data:/data
environment:
- MODEL_ID=ISOISS/jina-embeddings-v3-tei
- AUTO_TRUNCATE=true
runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
Error:
Error: Could not create backend
Caused by:
Could not start backend: cannot find tensor embeddings.word_embeddings.weight
Has anyone encountered this issue or knows how to fix it? Thanks in advance.