Apply for community grant: Academic project (gpu and storage)

#1
by potamides - opened
Natural Language Learning Group org

Creating high-quality scientific figures can be time-consuming and challenging, even though sketching ideas on paper is relatively easy. Furthermore, recreating existing figures that are not stored in formats preserving semantic information is equally complex. To tackle this problem, we introduce DeTikZify, a novel multimodal language model that automatically synthesizes scientific figures as semantics-preserving TikZ graphics programs based on sketches and existing figures. We also introduce an MCTS-based inference algorithm that enables DeTikZify to iteratively refine its outputs without the need for additional training.

DeTikZify has seen considerable interest from the LLM and LaTeX communities and beyond. A GPU upgrade could help us to make the model available to anyone (for successful inference a minimum of 20gb of VRAM is required, therefore the recommended minimum hardware would be an Nvidia A10G - small). Persistent storage would help us to permanently cache model downloads which can take a lot of time, but this is optional as it would not affect functionality. Thank you for considering our project for a community grant.

Hi @potamides , we've assigned 1xL4 for now, but would it be possible to migrate your Space to a normal Gradio Space? We recently started using ZeroGPU (docs) as the default hardware for grants, but it's only compatible with gradio SDK and doesn't work with docker Space.

Natural Language Learning Group org

Thank you so much! I wasn't aware of the L4 as an option, but it should work fine judging from the specs. Now testing it. I would love to use a normal Gradio space as it would save us the hassle of having to maintain our own docker image. The problem with that is, that we need access to a full TeX Live 2023 installation. While there is an option to install system packages with a normal Gradio Space, the version of TeX Live in the repositories is way to old for our model to work correctly (same problem with our previous project AutomaTikZ). If there is an option to install requirements from 3rd party sources (we need to run this script to be able to install the exact version of TeX Live we need), I would be happy to migrate.

Ah, I see. As it's not possible to install system packages that are not available via apt in a normal Gradio Space, I think we should keep this Space as is and use L4, then. Thanks for the explanation!

Sign up or log in to comment