awq_inference_engine.cpython-39-x86_64-linux-gnu.so: undefined symbol
#1
by
n094t23g
- opened
File "/home/s/anaconda3/envs/local_llms/lib/python3.9/site-packages/awq/modules/linear.py", line 4, in <module>
import awq_inference_engine # with CUDA kernels
ImportError: /home/s/anaconda3/envs/local_llms/lib/python3.9/site-packages/awq_inference_engine.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops15sum_dim_IntList4callERKNS_6TensorEN3c1016OptionalArrayRefIlEEbNS5_8optionalINS5_10ScalarTypeEEE
Thanks for the model.
Do I need a different version of cuda or pytorch for this to work?