How did you merge the decoder and decoder_with_past_model files?
#2
by
PROMCRdog
- opened
When converting seq-to-seq models to onnx, Optimum outputs two decoder files decoder_model.onnx and decoder_with_past_model, I understand one has prewritten keys to improve inference. But How can these two files be used with the encoder.onnx and is both needed or how can they be merged?
Hi there! I converted the models using Hugging Face's Optimum library: https://github.com/huggingface/optimum, which does the merging for you.
Here's the function which does the merging: https://github.com/huggingface/optimum/blob/f4809307e409d5ce698364ad48b69d38e0c406e9/optimum/onnx/graph_transformations.py#L197-L323