PygmalionAI

/

pygmalion-6b

like 725

I'm new to this, just trying to get started using the model on SageMaker, using the new Deploy to SageMaker function/script.

After copying across, and starting an inference endpoint (using the supplied code) I get the following error when trying to run:

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
"code": 400,
"type": "InternalServerException",
"message": "Could not load model /.sagemaker/mms/models/PygmalionAI__pygmalion-6b with any of the following classes: (\u003cclass \u0027transformers.models.auto.modeling_auto.AutoModelForSeq2SeqLM\u0027\u003e, \u003cclass \u0027transformers.models.auto.modeling_auto.AutoModelForCausalLM\u0027\u003e, \u003cclass \u0027transformers.models.gptj.modeling_gptj.GPTJForCausalLM\u0027\u003e)."
}

This is the input, as supplied in the deploy script:

predictor.predict({
'inputs': {
    "past_user_inputs": ["Which movie is the best ?"],
    "generated_responses": ["It's Die Hard for sure."],
    "text": "Can you explain why ?"
}
})

Cloudwatch logs:

[INFO ] W-PygmalionAI__pygmalion-6b-2-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: Could not load model /.sagemaker/mms/models/PygmalionAI__pygmalion-6b with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForSeq2SeqLM'>, <class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.gptj.modeling_gptj.GPTJForCausalLM'>). : 400