File size: 5,841 Bytes
055fea6
 
 
 
 
 
 
 
9fc2dc9
0e407c8
d4afeb7
 
bfb73a4
e673457
cbbce32
 
bfb73a4
06d3ffa
897989b
 
bfb73a4
06d3ffa
b62fb09
 
be70e70
e3b8b90
be70e70
b62fb09
c0bb2d2
20b1101
9dfde9a
d4afeb7
 
 
 
 
 
 
 
 
9aa91fc
fdec844
20b1101
 
d4afeb7
 
 
 
0613780
d4afeb7
 
3b19079
d4afeb7
fdec844
 
0b53a03
0613780
fdec844
d4afeb7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
b7ad1d4
fdec844
 
a56d2da
 
 
 
 
 
 
 
 
 
f98389c
 
 
a56d2da
f98389c
a56d2da
f98389c
 
 
 
 
 
 
 
 
613a4df
b62fb09
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
---
title: README
emoji: 🌍
colorFrom: indigo
colorTo: indigo
sdk: static
pinned: false
---
<div style="height: 15px;"></div>

# ZeroGPU Spaces

<div style="background-color: rgba(255, 255, 255, 0.5); color: rgba(0, 0, 0, 0.87); border: 1px solid rgba(0, 0, 0, 0.5); border-radius: 8px; padding: 0.5rem 1rem;">
  <span style="font-weight: 600;">ZeroGPU is currently in beta.</span> It's available to use for everyone. <a target="_blank" href="https://huggingface.co/subscribe/pro" style="color: inherit;">PRO</a> users or <a target="_blank" href="https://huggingface.co/enterprise" style="color: inherit;">Enterprise</a> organizations can host their own ZeroGPU Spaces under their namespaces.
</div>

<div style="background-color: rgba(255, 255, 255, 0.5); color: rgba(0, 0, 0, 0.87); border: 1px solid rgba(0, 0, 0, 0.5); border-radius: 8px; padding: 0.5rem 1rem; margin-top: 10px;">
  <span style="font-weight: 600;">Please share your feedback about ZeroGPU</span> in the <a target="_blank" href="https://huggingface.co/spaces/zero-gpu-explorers/README/discussions" style="color: inherit;">Community tab</a>.
</div>

<div style="background-color: rgba(255, 255, 255, 0.5); color: rgba(0, 0, 0, 0.87); border: 1px solid rgba(0, 0, 0, 0.5); border-radius: 8px; padding: 0.5rem 1rem; margin-top: 10px;">
  <span style="font-weight: 600;">Browse ZeroGPU Spaces</span> here: <a target="_blank" href="https://huggingface.co/spaces/enzostvs/zero-gpu-spaces" style="color: inherit;">Dedicated list</a>.
</div>

<div style="background-color: rgba(255, 255, 255, 0.5); color: rgba(0, 0, 0, 0.87); border: 1px solid rgba(0, 0, 0, 0.5); border-radius: 8px; padding: 0.5rem 1rem; margin-top: 10px;">
  <span style="font-weight: 600;">ZeroGPU works great with Dev Mode.</span> Learn more about <a target="_blank" href="https://huggingface.co/spaces/dev-mode-explorers/README" style="color: inherit;">Dev Mode</a>.
</div>

<img src="https://cdn-uploads.huggingface.co/production/uploads/5f17f0a0925b9863e28ad517/cAlvAOu9QC547zrmRVpS5.gif" style="width: 100%; max-width:950px"/>


*ZeroGPU* is a new kind of hardware for Spaces.

It has two goals :
- Provide **free GPU access** for Spaces
- Allow Spaces to run on **multiple GPUs**

This is achieved by making Spaces efficiently hold and release GPUs as needed
(as opposed to a classical GPU Space that holds exactly one GPU at any point in time)

ZeroGPU uses _Nvidia A100_ GPU devices under the hood (40GB of vRAM are available for each workloads)

<img src="https://cdn-uploads.huggingface.co/production/uploads/5f17f0a0925b9863e28ad517/naVZI-v41zNxmGlhEhGDJ.gif" style="width: 100%; max-width:550px"/>


# Compatibility

*ZeroGPU* Spaces should mostly be compatible with any PyTorch-based GPU Space.<br>
Compatibility with high level HF libraries like `transformers` or `diffusers` is slightly more guaranteed<br>
That said, ZeroGPU Spaces are not as broadly compatible as classical GPU Spaces and you might still encounter unexpected bugs

Also, for now, ZeroGPU Spaces only works with the **Gradio SDK**

Supported versions:
- Gradio: 4+
- PyTorch: All versions from `2.0.0` to `2.2.0`
- Python: `3.10.13`

# Usage

In order to make your Space work with ZeroGPU you need to **decorate** the Python functions that actually require a GPU with `@spaces.GPU`<br>
During the time when a decorated function is invoked, the Space will be attributed a GPU, and it will release it upon completion of the function.<br>
Here is a practical example :

```diff
+import spaces
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(...)
pipe.to('cuda')

+@spaces.GPU
def generate(prompt):
    return pipe(prompt).images

gr.Interface(
    fn=generate,
    inputs=gr.Text(),
    outputs=gr.Gallery(),
).launch()
```

1. We first `import spaces` (importing it first might prevent some issues but is not mandatory)
2. Then we decorate the `generate` function by adding a `@spaces.GPU` line before its definition

Note that `@spaces.GPU` is effect-free and can be safely used on non-ZeroGPU environments

## Duration

If you expect your GPU function to take more than __60s__ then you need to specify a `duration` param in the decorator like:

```python
@spaces.GPU(duration=120)
def generate(prompt):
   return pipe(prompt).images
```

It will set the maximum duration of your function call to 120s.  
You can also specify a duration if you know that your function will take far less than the 60s default.  
The lower the duration, the higher priority your Space visitors will have in the queue

## Getting Started

To **explore and use existing ZeroGPU Spaces**, browse the following list: https://huggingface.co/spaces/enzostvs/zero-gpu-spaces

To **create and host your own ZeroGPU Spaces**:

  - For personal accounts:
    - [Subscribe to PRO](https://huggingface.co/settings/billing/subscription). Once subscribed, ZeroGPU will appear in the hardware list on [New Space page](https://huggingface.co/new-space) when you select the Gradio SDK.
  - For organizations:
    - [Subscribe to the Enterprise](https://huggingface.co/enterprise). This will enable you to host and collaborate on ZeroGPU Spaces with all your organization members.
  
<div style="height: 35px;"></div>
<style>
  :root {
    --bg-color-light: rgba(255, 255, 255, 0.3);
    --bg-color-dark: rgba(0, 0, 0, 0.3);
    --text-color-light: rgb(0, 0, 0);
    --text-color-dark: rgb(255, 255, 255);
  }

  .theme-light {
    --bg-color: var(--bg-color-light);
    --text-color: var(--text-color-light);
  }

  .theme-dark {
    --bg-color: var(--bg-color-dark);
    --text-color: var(--text-color-dark);
  }

  .notice {
    background-color: var(--bg-color);
    color: var(--text-color);
    border-radius: 8px;
    padding: 0.5rem 1rem;
    margin-top: 10px;
  }

  .notice span {
    font-weight: 600;
  }
</style>