Unable to load the model through sentence_transformerss
I am on python 3.11.9, I have all the relevant libraries installed, but when I try to load the model, I run into the following exception:
>>> from sentence_transformers import SentenceTransformer
>>> model = SentenceTransformer("jinaai/jina-embeddings-v3", trust_remote_code=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/miniconda3/envs/ner/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 294, in __init__
modules, self.module_kwargs = self._load_sbert_model(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/ner/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1647, in _load_sbert_model
module = module_class(model_name_or_path, cache_dir=cache_folder, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.cache/huggingface/modules/transformers_modules/jinaai/jina-embeddings-v3/8676923fd77c70cb23b45bce9cc12a86f1e85ddf/custom_st.py", line 57, in __init__
self.auto_model = AutoModel.from_pretrained(model_name_or_path, config=self.config, cache_dir=cache_dir, **model_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/ner/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 551, in from_pretrained
model_class = get_class_from_dynamic_module(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/ner/lib/python3.11/site-packages/transformers/dynamic_module_utils.py", line 514, in get_class_from_dynamic_module
return get_class_in_module(class_name, final_module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/miniconda3/envs/ner/lib/python3.11/site-packages/transformers/dynamic_module_utils.py", line 212, in get_class_in_module
module_spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/user/.cache/huggingface/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/ffd672dfd02dc2a89ae410830eb223176405f950/modeling_lora.py", line 15, in <module>
from .modeling_xlm_roberta import (XLMRobertaFlashConfig, XLMRobertaModel,
File "/home/user/.cache/huggingface/modules/transformers_modules/jinaai/xlm-roberta-flash-implementation/ffd672dfd02dc2a89ae410830eb223176405f950/modeling_xlm_roberta.py", line 34, in <module>
from .block import Block
ModuleNotFoundError: No module named 'transformers_modules.jinaai.xlm-roberta-flash-implementation.ffd672dfd02dc2a89ae410830eb223176405f950.block'
Is there something I am missing?
Edit:
I have also tried the following:
- Download the files and load the model locally: same issue
- Create a new environment, install the libraries and then try loading the model: same issue
For some reason it seems to be working fine on colab, which is very strange to me.
I meet the same problem with python=3.10
Sometimes HF does this strange thing, only downloading the files that are imported in the model file. I believe that's what's happening here. So I've added some imports to the main file and it should be working fine now. Lmk if it works for you.
Yes, this seems to work fine now!
On a side note; I want to load the model locally, and not through huggingface hub every time. The workaround I have found for this is to have all the model files for jinaai/jina-embeddings-v3 and jinaai/xlm-roberta-flash-implementation, change the config.json of the embedding model to point to the local roberta configs, and then load it. It seems to work fine, is there a better, less convoluted way to achieve this?
Looks like it's resolved so I'm closing the issue
I'm still having this error, even after removing the cache. Anyone has a clue where it may come from?
maybe the best thing is move xlm-roberta-implementation to this repo to make everything easier @jupyterjazz
@tbaz
I removed the cache, and it didn't work, it worked when I added trust_remote_code=True. Funny huh?