cannot import name 'NVLM_D' from 'transformers'

by rohan13 - opened Oct 3

Discussion

rohan13

Oct 3

Is this working yet? I tried to run it using transformers latest version and I can't run inference, any suggestions?

boxin-wbx

NVIDIA org Oct 3

Hi @rohan13 ,

Please follow the README example and load model in this way:

path = "nvidia/NVLM-D-72B"
device_map = split_model()
model = AutoModel.from_pretrained(
    path,
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    use_flash_attn=False,
    trust_remote_code=True,
    device_map=device_map).eval()

Note that trust_remote_code=True is important as the modeling code is hosted in this repo.

Thanks,
Boxin

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment