Failed to deploy the model in inference endpoint with NO error

#25

by Isgservices-builderai - opened Mar 8

Discussion

Isgservices-builderai

Mar 8

•

edited Mar 8

Have tried to create an inference point for this model couple of time and it failed on all occasions without any error. Here is the snippet of my log:
What could be the issue? I'm using Nvidia A100 - 2xGPU.160GB. Have increased Max Number of Tokens to 8K.

Cc @zqh11 - @jxji

fs-tom

25 days ago

Same result Nvidia L4 · 4x GPU · 96 GB

fs-tom

25 days ago

Have tried to create an inference point for this model couple of time and it failed on all occasions without any error. Here is the snippet of my log:
What could be the issue? I'm using Nvidia A100 - 2xGPU.160GB. Have increased Max Number of Tokens to 8K.

Cc @zqh11 - @jxji

I got Bin12345/AutoCoder running

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment