Nyamdavaa Amar
Pipeline Parallelism with Controllable Memory
3d4d40d
|
raw
history blame
No virus
408 Bytes
## Alternative schedules
By utilizing the building block, we can search for different types of schedules depending on the need. We illustrate few of them here below:
* 1F1B-V schedule without doing any B-W split.
* Schedule with 2/3rd 1F1B memory by utilising B-W split. Note that two microbatches are included in a single building block to avoid collision.
* Variation of interleaved 1F1B with lower memory