About multi gpu for cell classification finetune
#300
by
Eyulaochen
- opened
Hi Dr. Christina,
I don't know how to finetune cell classification model in jupyter notebook example with multi gpus, since it need lots of gpu memory.
Best regards,
Huayang
Thank you for your interest in Geneformer and your question! Please see the pretraining example for one way you can distribute training using Deepspeed. You would apply something analogous for fine-tuning, modifying the script as needed for fine-tuning. Hugging Face has a lot of built in methods for distribution with their trainer which we would also suggest you check out.
ctheodoris
changed discussion status to
closed
Thanks for your reply. I found that just use
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "gpu ids"
Best regards,
Huayang