Wi-zz

/

joy-caption-pre-alpha

like 72

Could you please try the app-multi-alpha.py file in the main repo to see if multi-gpu is working for you. Usage is the same.

There's a bug in the multi script. Line 81

with torch.amp.autocast_mode.autocast(rank, enabled=True)

Should be:

with torch.amp.autocast_mode.autocast(device_type='cuda', enabled=True):

Otherwise it tries to assign the integer of rank to device_type which expects a string.

There's a bug in the multi script. Line 81

with torch.amp.autocast_mode.autocast(rank, enabled=True)

Should be:

with torch.amp.autocast_mode.autocast(device_type='cuda', enabled=True):

Otherwise it tries to assign the integer of rank to device_type which expects a string.

Thank you, I've updated the script.

Hi. I tested this having a 4060ti and a RX560 , CUDA working fine for other apps. Here is the output:

Loading image adapter 🖼️ on GPU 0
[rank0]: Traceback (most recent call last):
[rank0]: File "/home/se/Work_Disk/Apps/Joy-Caption/joy-caption-pre-alpha/app-multi-alpha.py", line 210, in
[rank0]: main()
[rank0]: File "/home/se/Work_Disk/Apps/Joy-Caption/joy-caption-pre-alpha/app-multi-alpha.py", line 207, in main
[rank0]: run(0, 1, args)
[rank0]: File "/home/se/Work_Disk/Apps/Joy-Caption/joy-caption-pre-alpha/app-multi-alpha.py", line 175, in run
[rank0]: models = load_models(rank)
[rank0]: File "/home/se/Work_Disk/Apps/Joy-Caption/joy-caption-pre-alpha/app-multi-alpha.py", line 60, in load_models
[rank0]: image_adapter.load_state_dict(torch.load(CHECKPOINT_PATH / "image_adapter.pt", map_location=f"cuda:{rank}", weights_only=True))
[rank0]: File "/home/se/Work_Disk/Apps/Joy-Caption/joy-caption-pre-alpha/venv/lib/python3.9/site-packages/torch/serialization.py", line 1113, in load
[rank0]: raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
[rank0]: _pickle.UnpicklingError: Weights only load failed. Re-running torch.load with weights_only set to False will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
[rank0]: Please file an issue with the following so that we can make weights_only=True compatible with your use case: WeightsUnpickler error: Unsupported operand 118

[rank0]: Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.

Hope it helps.