anvilarth commited on
Commit
9f7f03a
β€’
1 Parent(s): 53ccd2b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -133
README.md CHANGED
@@ -1,133 +1,8 @@
1
- <div align="center">
2
-
3
- <img src="examples/logo.svg" width="130em" />
4
-
5
- # Garage: Generative Augmentation Framework for Transforming Object Representations in Images
6
-
7
- ![Project Page](examples/AbstractGARAGE.jpeg)
8
- </div>
9
-
10
- Welcome to **Garage**, a cutting-edge Python library designed for generative image augmentation! Our library includes the Augmenter model, which leverages advanced machine learning techniques to generate new objects and prompts, seamlessly replacing specified objects in images with new ones using PowerPaint.
11
-
12
- ## Get Started
13
-
14
- ```bash
15
- # Clone the Repository
16
- git clone https://github.com/DorinDaniil/Garage.git
17
-
18
- # Create Virtual Environment with Conda
19
- conda create --name garage python=3.10
20
- conda activate garage
21
-
22
- # Install Dependencies
23
- pip install -r requirements.txt
24
-
25
- # Download the Required Weights
26
- bash checkpoints.sh
27
-
28
- # Required Dependencies for Demo
29
- python Garage/models/GroundedSegmentAnything/GroundingDINO/setup.py install
30
- ```
31
-
32
- ### Use Demo
33
- You can launch the Gradio interface for **Garage** by running the following command:
34
- ```bash
35
- conda activate garage
36
- python app.py --share
37
- ```
38
-
39
- Upon launching the interface, you can upload images to augment with new objects. You have the option to segment an object mask using a text prompt or by selecting a mask from examples. To choose a mask, simply click on it in the right frame. Next, enter the text prompt for the new object you wish to replace the segmented object with. Utilize the advanced options to adjust the segmentation and augmentation process.
40
- ![Demo](examples/demo_scheme_final.png)
41
-
42
- ### Use Docker
43
- Dockerfile authomatically starts demo app in file [app.py](app.py)
44
-
45
- To start with Docker run following commands:
46
- ```bash
47
- # build image
48
- docker build -t garage-app .
49
-
50
- # run container
51
- docker run -it --gpus all -p 7860:7860 --name garage garage-app
52
- ```
53
-
54
- To use demo app wait for the app to load and go to the following link in your browser:
55
- ```
56
- Running on local URL: http://0.0.0.0:7860
57
- ```
58
-
59
- ### Use Alpha-CLIP Filter
60
- To use Alpha-CLIP to filter augmentations, follow the instructions in the [alphaCLIP_filter](https://github.com/DorinDaniil/Garage/tree/main/alphaCLIP_filter) directory.
61
-
62
- ## Inference
63
- Here's a step-by-step guide on how to use the **Garage** library to perform image augmentation:
64
-
65
- ### Import the necessary modules
66
-
67
- ```python
68
- from Garage import Augmenter
69
- from PIL import Image
70
- ```
71
-
72
- ### Initialize the Augmenter class
73
-
74
- The Augmenter class is the main interface for performing image augmentation. You can initialize it with the device you want to use for computations, which defaults to "cuda".
75
-
76
- ```python
77
- augmenter = Augmenter(device="cuda")
78
- ```
79
-
80
- ### Prepare your inputs
81
-
82
- You will need to provide the following inputs to the Augmenter:
83
-
84
- - `image`: The input image in PIL format.
85
- - `mask`: The mask of the object to replace in PIL format.
86
- - `current_object`: The name of the object to be replaced.
87
- - `new_objects_list` (optional): A list of potential new objects. If None, the method will generate a new object.
88
- - `ddim_steps` (optional): The number of denoising steps. More steps mean a slower but potentially higher quality result. Defaults to 50.
89
- - `guidance_scale` (optional): The scale for classifier-free guidance. Higher values lead to results that are more closely linked to the text prompt. Defaults to 5.
90
- - `seed` (optional): Integer value that initializes the random number generator for reproducibility. Defaults to 1.
91
- - `return_prompt` (optional): If True, the method also returns the prompt used for generation and the new object. Defaults to False.
92
-
93
- ### Perform image augmentation
94
-
95
- You can perform image augmentation by calling the Augmenter instance with the prepared inputs:
96
-
97
- ```python
98
- image = Image.open("path/to/your/image.jpg")
99
- mask = Image.open("path/to/your/mask.jpg")
100
-
101
- result, (prompt, new_object) = augmenter(
102
- image=image,
103
- mask=mask,
104
- current_object="replacement object",
105
- new_objects_list=None,
106
- ddim_steps=50,
107
- guidance_scale=5,
108
- seed=1,
109
- return_prompt=True
110
- )
111
- ```
112
-
113
- The `result` variable will contain the modified image, prompt used for generation and the new object, respectively.
114
-
115
- ### Examples
116
-
117
- More examples of how to use **Garage** refer to the example.ipynb
118
-
119
- ## Download augmented datasets
120
-
121
- To download the [VOC2007 augmentations](https://huggingface.co/datasets/danulkin/VOC2007Augs), follow the instructions:
122
- ```python
123
- from huggingface_hub import snapshot_download
124
-
125
- snapshot_download(repo_id="danulkin/VOC2007Augs", repo_type="dataset", local_dir = "./VOC2007Augs")
126
- ```
127
-
128
- ## Contact Us
129
- **Andrei Filatov**: filatovandreiv@gmail.com
130
-
131
- **Daniil Dorin**: dorin.dd@phystech.edu
132
-
133
- **Ulyana Izmesteva**: izmesteva.ua@phystech.edu
 
1
+ title: Garage
2
+ emoji: πŸ“š
3
+ colorFrom: pink
4
+ colorTo: indigo
5
+ sdk: gradio
6
+ sdk_version: 4.40.0
7
+ app_file: app.py
8
+ pinned: false