python3: No module named lmdeploy.serve.turbomind.main;

by victorx98 - opened Sep 22, 2023

Discussion

victorx98

Sep 22, 2023

•

edited Sep 22, 2023

Hello,

I'm running command: python3 -m lmdeploy.serve.turbomind ./workspace --server_name 127.0.0.1 --server_port 5001
But got the error message:
python3: No module named lmdeploy.serve.turbomind.main; 'lmdeploy.serve.turbomind' is a package and cannot be directly executed

What could be wrong?

Btw, running the chat mode is really fast. Probably around 30 token /s on RTX 3090. Took about 23GB VRAM, despite the model file size only 12GB

Thanks

unsubscribe

InternLM org Oct 9, 2023

You are using a wrong script. Where did you get the command?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

python3: No module named lmdeploy.serve.turbomind.__main__;

python3: No module named lmdeploy.serve.turbomind.main;