vram+ram
#7
by
sdyy
- opened
iam ask
unsloth/Llama-3.1-Nemotron-70B-Instruct-bnb-4bit
run on vram 16gb
with
ram 32g
????????????????
iam ask
unsloth/Llama-3.1-Nemotron-70B-Instruct-bnb-4bit
run on vram 16gb
with
ram 32g????????????????
Too less vram not going to work
Does Unsloth not support CPU offloading?
Does it support accelerae library to run it with devicemap auto or not
Does Unsloth not support CPU offloading?
We support it yes but 16GB VRAM is just way too less