How did you merge the decoder and decoder_with_past_model files?

#2
by PROMCRdog - opened

When converting seq-to-seq models to onnx, Optimum outputs two decoder files decoder_model.onnx and decoder_with_past_model, I understand one has prewritten keys to improve inference. But How can these two files be used with the encoder.onnx and is both needed or how can they be merged?

ONNX Community org

Hi there! I converted the models using Hugging Face's Optimum library: https://github.com/huggingface/optimum, which does the merging for you.

Here's the function which does the merging: https://github.com/huggingface/optimum/blob/f4809307e409d5ce698364ad48b69d38e0c406e9/optimum/onnx/graph_transformations.py#L197-L323

Sign up or log in to comment