Error with Text-Embeddings-Inference Container using JINA-v3 Model

#1
by ilhamdprastyo - opened

I'm encountering an error while running text-embeddings-inference with JINA-v3 model on a single NVIDIA A6000 GPU. Here's my setup and error:

Docker-compose:

version: '3.8'
services:
  text-embeddings-inference-jina-v3:
      image: ghcr.io/huggingface/text-embeddings-inference:latest
      container_name: text_embeddings_inference_jina_v3
      ports:
        - "8083:80"
      restart: always
      volumes:
        - /data:/data
      environment:
        - MODEL_ID=ISOISS/jina-embeddings-v3-tei
        - AUTO_TRUNCATE=true
      runtime: nvidia
      deploy:
        resources:
          reservations:
            devices:
              - driver: nvidia
                count: 1
                capabilities: [gpu]

Error:

Error: Could not create backend
Caused by:
    Could not start backend: cannot find tensor embeddings.word_embeddings.weight

Has anyone encountered this issue or knows how to fix it? Thanks in advance.

Sign up or log in to comment