Update README.md
Browse files
README.md
CHANGED
@@ -1,133 +1,8 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
Welcome to **Garage**, a cutting-edge Python library designed for generative image augmentation! Our library includes the Augmenter model, which leverages advanced machine learning techniques to generate new objects and prompts, seamlessly replacing specified objects in images with new ones using PowerPaint.
|
11 |
-
|
12 |
-
## Get Started
|
13 |
-
|
14 |
-
```bash
|
15 |
-
# Clone the Repository
|
16 |
-
git clone https://github.com/DorinDaniil/Garage.git
|
17 |
-
|
18 |
-
# Create Virtual Environment with Conda
|
19 |
-
conda create --name garage python=3.10
|
20 |
-
conda activate garage
|
21 |
-
|
22 |
-
# Install Dependencies
|
23 |
-
pip install -r requirements.txt
|
24 |
-
|
25 |
-
# Download the Required Weights
|
26 |
-
bash checkpoints.sh
|
27 |
-
|
28 |
-
# Required Dependencies for Demo
|
29 |
-
python Garage/models/GroundedSegmentAnything/GroundingDINO/setup.py install
|
30 |
-
```
|
31 |
-
|
32 |
-
### Use Demo
|
33 |
-
You can launch the Gradio interface for **Garage** by running the following command:
|
34 |
-
```bash
|
35 |
-
conda activate garage
|
36 |
-
python app.py --share
|
37 |
-
```
|
38 |
-
|
39 |
-
Upon launching the interface, you can upload images to augment with new objects. You have the option to segment an object mask using a text prompt or by selecting a mask from examples. To choose a mask, simply click on it in the right frame. Next, enter the text prompt for the new object you wish to replace the segmented object with. Utilize the advanced options to adjust the segmentation and augmentation process.
|
40 |
-
![Demo](examples/demo_scheme_final.png)
|
41 |
-
|
42 |
-
### Use Docker
|
43 |
-
Dockerfile authomatically starts demo app in file [app.py](app.py)
|
44 |
-
|
45 |
-
To start with Docker run following commands:
|
46 |
-
```bash
|
47 |
-
# build image
|
48 |
-
docker build -t garage-app .
|
49 |
-
|
50 |
-
# run container
|
51 |
-
docker run -it --gpus all -p 7860:7860 --name garage garage-app
|
52 |
-
```
|
53 |
-
|
54 |
-
To use demo app wait for the app to load and go to the following link in your browser:
|
55 |
-
```
|
56 |
-
Running on local URL: http://0.0.0.0:7860
|
57 |
-
```
|
58 |
-
|
59 |
-
### Use Alpha-CLIP Filter
|
60 |
-
To use Alpha-CLIP to filter augmentations, follow the instructions in the [alphaCLIP_filter](https://github.com/DorinDaniil/Garage/tree/main/alphaCLIP_filter) directory.
|
61 |
-
|
62 |
-
## Inference
|
63 |
-
Here's a step-by-step guide on how to use the **Garage** library to perform image augmentation:
|
64 |
-
|
65 |
-
### Import the necessary modules
|
66 |
-
|
67 |
-
```python
|
68 |
-
from Garage import Augmenter
|
69 |
-
from PIL import Image
|
70 |
-
```
|
71 |
-
|
72 |
-
### Initialize the Augmenter class
|
73 |
-
|
74 |
-
The Augmenter class is the main interface for performing image augmentation. You can initialize it with the device you want to use for computations, which defaults to "cuda".
|
75 |
-
|
76 |
-
```python
|
77 |
-
augmenter = Augmenter(device="cuda")
|
78 |
-
```
|
79 |
-
|
80 |
-
### Prepare your inputs
|
81 |
-
|
82 |
-
You will need to provide the following inputs to the Augmenter:
|
83 |
-
|
84 |
-
- `image`: The input image in PIL format.
|
85 |
-
- `mask`: The mask of the object to replace in PIL format.
|
86 |
-
- `current_object`: The name of the object to be replaced.
|
87 |
-
- `new_objects_list` (optional): A list of potential new objects. If None, the method will generate a new object.
|
88 |
-
- `ddim_steps` (optional): The number of denoising steps. More steps mean a slower but potentially higher quality result. Defaults to 50.
|
89 |
-
- `guidance_scale` (optional): The scale for classifier-free guidance. Higher values lead to results that are more closely linked to the text prompt. Defaults to 5.
|
90 |
-
- `seed` (optional): Integer value that initializes the random number generator for reproducibility. Defaults to 1.
|
91 |
-
- `return_prompt` (optional): If True, the method also returns the prompt used for generation and the new object. Defaults to False.
|
92 |
-
|
93 |
-
### Perform image augmentation
|
94 |
-
|
95 |
-
You can perform image augmentation by calling the Augmenter instance with the prepared inputs:
|
96 |
-
|
97 |
-
```python
|
98 |
-
image = Image.open("path/to/your/image.jpg")
|
99 |
-
mask = Image.open("path/to/your/mask.jpg")
|
100 |
-
|
101 |
-
result, (prompt, new_object) = augmenter(
|
102 |
-
image=image,
|
103 |
-
mask=mask,
|
104 |
-
current_object="replacement object",
|
105 |
-
new_objects_list=None,
|
106 |
-
ddim_steps=50,
|
107 |
-
guidance_scale=5,
|
108 |
-
seed=1,
|
109 |
-
return_prompt=True
|
110 |
-
)
|
111 |
-
```
|
112 |
-
|
113 |
-
The `result` variable will contain the modified image, prompt used for generation and the new object, respectively.
|
114 |
-
|
115 |
-
### Examples
|
116 |
-
|
117 |
-
More examples of how to use **Garage** refer to the example.ipynb
|
118 |
-
|
119 |
-
## Download augmented datasets
|
120 |
-
|
121 |
-
To download the [VOC2007 augmentations](https://huggingface.co/datasets/danulkin/VOC2007Augs), follow the instructions:
|
122 |
-
```python
|
123 |
-
from huggingface_hub import snapshot_download
|
124 |
-
|
125 |
-
snapshot_download(repo_id="danulkin/VOC2007Augs", repo_type="dataset", local_dir = "./VOC2007Augs")
|
126 |
-
```
|
127 |
-
|
128 |
-
## Contact Us
|
129 |
-
**Andrei Filatov**: filatovandreiv@gmail.com
|
130 |
-
|
131 |
-
**Daniil Dorin**: dorin.dd@phystech.edu
|
132 |
-
|
133 |
-
**Ulyana Izmesteva**: izmesteva.ua@phystech.edu
|
|
|
1 |
+
title: Garage
|
2 |
+
emoji: π
|
3 |
+
colorFrom: pink
|
4 |
+
colorTo: indigo
|
5 |
+
sdk: gradio
|
6 |
+
sdk_version: 4.40.0
|
7 |
+
app_file: app.py
|
8 |
+
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|