facebook/seamless_m4t · How to do batch reasoning?

Hey @stu-lupeng - could you clarify what you mean by 'batch reasoning'? As in 'batched inference'? The Translator class provided in the seamless communication repository only supports batch size =1: inference/translator

You can use a lower-level API from the seamless communication repo if you want to explore running batched inference. The predict function looks like it has the workings of a function that could generalise to being batched: inference/translator/predict

I would use this as a starting point! For reference, the transformers integration of Seamless M4T is well underway with @ylacombe . This will support batched inference out of the box.