THUDM/cogvlm2-llama3-chat-19B · cannot be used on Apple MPS

May 23

unfortunately due to the reliance on OpenAI's Triton library as well as Xformers this thing is effectively unable to run on Apple hardware:

  File "/Users/bghira/.cache/huggingface/modules/transformers_modules/THUDM/cogvlm-chat-hf/e29dc3ba206d524bf8efbfc60d80fc4556ab0e3c/modeling_cogvlm.py", line 19, in <module>
    from .visual import EVA2CLIPModel
  File "/Users/bghira/.cache/huggingface/modules/transformers_modules/THUDM/cogvlm-chat-hf/e29dc3ba206d524bf8efbfc60d80fc4556ab0e3c/visual.py", line 4, in <module>
    import xformers.ops as xops

bghira

May 23

note that this applies to v1.1 and v2.0 of CogVLM.

zRzRzRzRzRzRzR

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org May 24

need use Linux with Nvidia GPU