Describe CrossEncoder integration with Sentence Transformers
Hello!
Pull Request overview
- Describe CrossEncoder integration with Sentence Transformers
Details
Because CrossEncoder already works directly with AutoModelForSequenceClassification, it works out of the box. You've got some nice extra behaviour for your rerank method though, so I've placed the Sentence Transformers support below the transformers usage.
Congratulations on the release, I think I'll be recommending this model on my GitHub issues. At least, for those for who the license isn't an issue.
- Tom Aarsen
Hi Tom, thanks for your contribution!
We have tested the capability with sentence-transformer. The model.predict() works pretty well and returns the desired results. However, model.rerank() does not work since we use bfloat16 and sentence-transformer will return the ranking score as numpy list which doesn't support bfloat16.
We will raise a PR in sentence-transformer to fix this issue. And we will merge your PR once the issue is resolved in sentence-transformer.
Thanks again!
Hello!
I've encountered the same, and so has one of my users: https://github.com/UKPLab/sentence-transformers/pull/2783
I'll be merging
@malteos
his PR, and then users can use both predict and rank with the default parameters (e.g. no need for convert_to_tensor=True) when installing the bleeding-edge version of Sentence Transformers.
Until then, I've added convert_to_tensor=True to the rank method. This will cause the method to keep the outputs as bfloat16 rather than convert them to numpy, and it avoids the problem altogether. I think this PR should be ready to go now :)
- Tom Aarsen
LGTM, thanks for your work.