tiiuae

/

falcon-7b-instruct

like 879

Hi I fine tuned the model using this tutorial. It works great in notebook.
https://colab.research.google.com/drive/1FxlUb_H6Xirhkx4RszAgHeb2uDW7oKIH

After I deploy to inference endpoint, i get this error: "nRuntimeError: weight transformer.word_embeddings.weight does not exist"

Could someone please advise how to fix?

To replicate the issue, you could try deploying this model here:
https://huggingface.co/vrsen/falcon-7b-instruct-ft

You will see the same failure that I see. Could someone please help?

I am following these tutorials:
https://www.youtube.com/watch?v=AXG7TA7vIQ8&t=194s&ab_channel=VRSEN
https://www.youtube.com/watch?v=VdKdQYduGQc&ab_channel=VRSEN

more logs:
RuntimeError(f"weight {tensor_name} does not exist")\nRuntimeError: weight transformer.word_embeddings.weight does not exist\n"},"target":"text_generation_launcher","span":{"rank":0,"name":"shard-manager"},"spans":[{"rank":0,"name":"shard-manager"}]}
526bf 2023-07-16T20:37:19.492Z {"timestamp":"2023-07-16T20:37:19.491918Z","level":"INFO","fields":{"message":"Shutting down shards"},"target":"text_generation_launcher"}

526bf 2023-07-16T20:37:19.492Z {"timestamp":"2023-07-16T20:37:19.491898Z","level":"ERROR","fields":{"message":"You are using a model of type RefinedWebModel to instantiate a model of type . This is not supported for all configurations of models and can yield errors.\nTraceback (most recent call last):\n\n File

line 49, in get_filename\n raise RuntimeError(f"weight {tensor_name} does not exist")\n\nRuntimeError: weight transformer.word_embeddings.weight does not exist\n\n"},"target":"text_generation_launcher"}
526bf 2023-07-16T20:37:19.492Z Error: ShardCannotStart
526bf 2023-07-16T20:37:19.492Z {"timestamp":"2023-07-16T20:37:19.491861Z","level":"ERROR","fields":{"message":"Shard 0 failed to start"},"target":"text_generation_launcher"}