Ensuring compatibility with the sentence-transformers library

#24

I encountered the following error when attempting to load multilingual-e5-large-instruct using the sentence-transformers library:
not found: multilingual-e5-large-instruct/sentence_xlnet_config.json.

Upon investigating, I noticed that the sentence-transformers library looks for additional model configuration files like sentence_bert_config.json during model loading, as shown in this code snippet:
(https://github.com/UKPLab/sentence-transformers/blob/c68bf68299a4435c6a48ea15d789fef596bf1444/sentence_transformers/models/Transformer.py#L527-L540)

Additionally, other embedding models, such as bge-m3, also include this configuration file: https://huggingface.co/BAAI/bge-m3/blob/main/sentence_bert_config.json

To address this issue, I created the necessary sentence_bert_config.json file based on the xlm-roberta configuration.

Hi, thanks for reporting!

But I could not reproduce this issue using the latest version sentence_transformers==3.4.1.

I am aware some older versions of SentenceTransformers might look for this file, so I'll merge this pull request for now.

intfloat changed pull request status to merged
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment