Do you support vllm embedding?
Tried to run vllm server with the parameter --task embed. when the client sends an embedding request, the server fails with the following exception:
INFO: ::1:48876 - "POST /v1/embeddings HTTP/1.1" 500 Internal Server Error
ERROR 02-16 12:47:24 engine.py:135] AttributeError("'JambaForEmbedding' object has no attribute 'lm_head'")
ERROR 02-16 12:47:24 engine.py:135] Traceback (most recent call last):
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 133, in start
ERROR 02-16 12:47:24 engine.py:135] self.run_engine_loop()
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 196, in run_engine_loop
ERROR 02-16 12:47:24 engine.py:135] request_outputs = self.engine_step()
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 214, in engine_step
ERROR 02-16 12:47:24 engine.py:135] raise e
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/engine/multiprocessing/engine.py", line 205, in engine_step
ERROR 02-16 12:47:24 engine.py:135] return self.engine.step()
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 1390, in step
ERROR 02-16 12:47:24 engine.py:135] outputs = self.model_executor.execute_model(
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/executor/gpu_executor.py", line 88, in execute_model
ERROR 02-16 12:47:24 engine.py:135] output = self.driver_worker.execute_model(execute_model_req)
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/worker/worker_base.py", line 343, in execute_model
ERROR 02-16 12:47:24 engine.py:135] output = self.model_runner.execute_model(
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 02-16 12:47:24 engine.py:135] return func(*args, **kwargs)
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/worker/pooling_model_runner.py", line 109, in execute_model
ERROR 02-16 12:47:24 engine.py:135] hidden_or_intermediate_states = model_executable(
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 02-16 12:47:24 engine.py:135] return self._call_impl(*args, **kwargs)
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 02-16 12:47:24 engine.py:135] return forward_call(*args, **kwargs)
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/vllm/model_executor/models/jamba.py", line 453, in forward
ERROR 02-16 12:47:24 engine.py:135] self.lm_head.weight.dtype, num_mamba_layers,
ERROR 02-16 12:47:24 engine.py:135] ^^^^^^^^^^^^
ERROR 02-16 12:47:24 engine.py:135] File "/workspace/.vllm_env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1931, in getattr
ERROR 02-16 12:47:24 engine.py:135] raise AttributeError(
ERROR 02-16 12:47:24 engine.py:135] AttributeError: 'JambaForEmbedding' object has no attribute 'lm_head'