Bad performance of bge-reranker-v2-gemma compare with bge-reranker-v2-m3

#22

by shaunxu - opened Jul 3

Jul 3

As described in README for better performance it's recommended to use bge-reranker-v2-gemma. But in my case it takes 20 - 30 seconds to compute scores while bge-reranker-v2-m3 only needs 0.5 seconds.
PS, I tested on my MacBook Pro M1 with cpu. Not sure if this is the case.

Shitao

Beijing Academy of Artificial Intelligence org Jul 3

Hi, @shaunxu , bge-reranker-v2-m3 is smaller than gemma, so it's faster than gemma. But for general capabilities, the larger model gemma may have better accuracy.

qinrong

Jul 4

Hi， Do you know hot wo depoy bge-reranker-v2-m3 on text-embeddings-inference

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment