facebook

/

seamless-m4t-v2-large

like 552

What errors are you facing? What hardware are you using?

What errors are you facing? What hardware are you using?

Is there a sample code for training and fine-tuning m4t v2?
Because UnitYNART2UModel should be trained
In this case, I face errors such as RuntimeError: mat1 and mat2 shapes cannot be multiplied.
There is no problem with training the version 1 and fine tuning is done and the interesting thing is that after training on the customized data, there is no change in the output of the model and it reads as before.
I am really tired and confused after 3 months of trying

Has anyone taught the v2 model?? And is this possible now? Please explain if you remember

i use a100 or v100 gpu 40gb
and no problem in this part

sample code for do this :

if isinstance(self.model.t2u_model, UnitYNART2UModel): ################## new FT

            unit_decoder_out, decoder_padding_mask, durations = self.model.t2u_model.forward(
                text_decoder_output=text_decoder_out,
                text_decoder_padding_mask=text_decoder_padding_mask,
                text_seqs=seqs,
            )
            print('unit_decoder_out.logits : ', unit_decoder_out.logits)

            unit_logits = self.model.t2u_model.final_proj(unit_decoder_out.logits)
            print('aaaaaaaaaaaaaaaa')
        else:
            raise NotImplementedError("T2U finetuning not implemented ")