转onnx。

#12
by angellee - opened

您好,我想将m3e-base模型转成onnx进行使用,pt文件在哪里呢?

Moka HR SaSS org

https://huggingface.co/moka-ai/m3e-base/blob/main/pytorch_model.bin

huggingface 的项目其实就是一个 git 项目,直接下载就可以了,git clone 这种。

您好,可以说一下具体怎么转吗?这个bin文件不是只有权重
参数吗?

Moka HR SaSS org

安装 optimum,执行下方代码即可

from optimum.onnxruntime import ORTModelForFeatureExtraction
from transformers import AutoTokenizer

model_checkpoint = "moka-ai/m3e-base"
save_directory = "m3e-base-onnx/"

# Load a model from transformers and export it to ONNX
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
ort_model = ORTModelForFeatureExtraction.from_pretrained(model_checkpoint, export=True, from_transformers=True)

# Save the ONNX model and tokenizer
ort_model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)

具体原理可以参考这个教程 https://huggingface.co/docs/optimum/quicktour#onnx-runtime

安装 optimum,执行下方代码即可

from optimum.onnxruntime import ORTModelForFeatureExtraction
from transformers import AutoTokenizer

model_checkpoint = "moka-ai/m3e-base"
save_directory = "m3e-base-onnx/"

# Load a model from transformers and export it to ONNX
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
ort_model = ORTModelForFeatureExtraction.from_pretrained(model_checkpoint, export=True, from_transformers=True)

# Save the ONNX model and tokenizer
ort_model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)

具体原理可以参考这个教程 https://huggingface.co/docs/optimum/quicktour#onnx-runtime

请问怎么推理呢?

安装 optimum,执行下方代码即可

from optimum.onnxruntime import ORTModelForFeatureExtraction
from transformers import AutoTokenizer

model_checkpoint = "moka-ai/m3e-base"
save_directory = "m3e-base-onnx/"

# Load a model from transformers and export it to ONNX
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
ort_model = ORTModelForFeatureExtraction.from_pretrained(model_checkpoint, export=True, from_transformers=True)

# Save the ONNX model and tokenizer
ort_model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)

具体原理可以参考这个教程 https://huggingface.co/docs/optimum/quicktour#onnx-runtime

你好,我用这段脚本成功得到了onnx的model,但是onnxruntime运行时报错了:
const T *onnxruntime::OpKernelContext::Input(int) const [T = onnxruntime::Tensor] Missing Input: token_type_ids
请问是什么原因呢?

Sign up or log in to comment