Would this run on a 32GM RAM & 8GB VRAM?
#14
by
Troyanovsky
- opened
Can it be loaded partially on GPU? Or do I have to load it all on RAM with CPU?
With a recent GPU (RTX), you can probably load it partially on GPU, juste use the "n-gpu-layers" option before loading.
Regarding your GPU, set it to a low value, and then increase it gradually if the VRAM isn't full