Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results
text-generation-inference
Inference Endpoints

Is is feasible to use this checkpoint for multi node inference via deepspeed Zero stage-3

#275
by YuTian8328 - opened

I'm trying to use deepseed Zero stage-3 to do multi node inference by loading this checkpoints, but got various problems. So, I was wondering if it is feasible at all?

Sign up or log in to comment