Apply for community grant: Personal project (gpu)

#1
by Tonic - opened
Tonic AI org

πŸ™‹πŸ»β€β™‚οΈ hey there HuggingFace Community,

I would like to apply for a community grant to keep Orca2 from Microsoft running & give folks the chance to test it out and use it via API in their applications.

Hope this helps !

-Tonic.

Hello @Tonic , we wanted to let you know that we've assigned a GPU to your space, and your GPU grant application has been approved. Congratulations! Please keep in mind that GPU grants are provided on a temporary basis and may be removed if usage is very low.
To learn more about GPUs in Spaces, please check out https://huggingface.co/docs/hub/spaces-gpus. We look forward to seeing the innovative work you produce with this grant. If you have any questions or concerns, please let us know. Thank you for your interest in our platform!

I have done a factory rebuild of the space as well.

Tonic AI org

ah okay! thanks for this much obliged <3

It is failing with OOM, can you please try implementing quantization? I have assigned an A10Large in the meanwhile with a 1 hour sleep. We can change the GPU back to T4 once we can bring down the memory usage.

Tonic AI org

yes, done, with all my thanks πŸ™πŸ»

Tonic AI org
This comment has been hidden
Tonic AI org

@ysharma got it to run on an A100 , sorry for the iterations : just in the middle of work + i hope i can catch you before the end of the work day ;-) πŸš€

Tonic AI org

@ysharma : solution i'm thinking for : duplicate my A100 space with orca to another space and use this grant for a mistral model that i know can fit + i can use immediately :-) hope that's okay (if community grant cant cover an A100 which i do understand )

Hi @Tonic , sorry for the delay in responding to your messages. Assigning an A100 for a GPU grant can be a complex process. Have you considered using quantization to reduce the memory usage of your application? Additionally, we have another demo available for Orca-2 that you might find helpful. You can access it here - https://huggingface.co/spaces/ari9dam/Orca-2-13B.
For another demo you'd have to apply for another grant request stating the project ideas in the request.

Tonic AI org
This comment has been hidden
Tonic AI org
This comment has been hidden
Tonic AI org
This comment has been hidden

Sign up or log in to comment