OOM

#7
by DAYTONE2903 - opened

How i can fix Oout of memory error .Device : rtx 4090 , VRAM : 24 GB
self.model = load_flow_model(self.name, device="cpu" if offload else self.torch_device)
File "/home/taras/flux/flux/util.py", line 344, in load_flow_model
sd = load_sft(ckpt_path, device=str(device))
File "/home/taras/flux/.venv/lib/python3.10/site-packages/safetensors/torch.py", line 315, in load_file
result[k] = f.get_tensor(k)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 90.00 MiB. GPU 0 has a total capacity of 23.63 GiB of which 96.25 MiB is free. Including non-PyTorch memory, this process has 23.35 GiB memory in use. Of the allocated memory 22.97 GiB is allocated by PyTorch, and 7.31 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.

Outpainting + 400px on an image with a resolution of 2336x1568px works well for me in ComfyUI
(RTX 3090 24GB, 32GB RAM) Workflow: https://pastebin.com/7fxwn3Am

Sign up or log in to comment