mpatel57 commited on
Commit
ea91f46
1 Parent(s): 90b49f4

inference code added

Browse files
Files changed (1) hide show
  1. README.md +50 -2
README.md CHANGED
@@ -12,6 +12,7 @@ tags:
12
  ---
13
 
14
  # Introduction
 
15
 
16
  The λ-ECLIPSE model is a light weight support for multi-concept personalization. λ-ECLIPSE is tiny T2I prior model designed for Kandinsky v2.2 diffusion image generator.
17
 
@@ -32,14 +33,61 @@ More examples at: [Gallery](https://eclipse-t2i.github.io/Lambda-ECLIPSE/gallery
32
 
33
  ## Installation
34
  ```bash
35
- git clone git@github.com:eclipse-t2i/eclipse-inference.git
36
  conda create -p ./venv python=3.9
37
  pip install -r requirements.txt
38
  ```
39
 
40
  ## Run Inference
 
41
 
42
- TBD
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
  ## Important Notes (and limitations):
45
 
 
12
  ---
13
 
14
  # Introduction
15
+ <a href="https://colab.research.google.com/drive/1VcqzXZmilntec3AsIyzCqlstEhX4Pa1o?usp=sharing" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
16
 
17
  The &lambda;-ECLIPSE model is a light weight support for multi-concept personalization. &lambda;-ECLIPSE is tiny T2I prior model designed for Kandinsky v2.2 diffusion image generator.
18
 
 
33
 
34
  ## Installation
35
  ```bash
36
+ git clone https://github.com/eclipse-t2i/lambda-eclipse-inference.git
37
  conda create -p ./venv python=3.9
38
  pip install -r requirements.txt
39
  ```
40
 
41
  ## Run Inference
42
+ <a href="https://colab.research.google.com/drive/1VcqzXZmilntec3AsIyzCqlstEhX4Pa1o?usp=sharing" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
43
 
44
+ ```bash
45
+ import os
46
+ import torch
47
+ from transformers import (
48
+ CLIPTextModelWithProjection,
49
+ CLIPTokenizer,
50
+ )
51
+ from src.pipelines.pipeline_kandinsky_subject_prior import KandinskyPriorPipeline
52
+ from src.priors.lambda_prior_transformer import PriorTransformer
53
+ from diffusers import DiffusionPipeline
54
+
55
+ text_encoder = CLIPTextModelWithProjection.from_pretrained(
56
+ "laion/CLIP-ViT-bigG-14-laion2B-39B-b160k",
57
+ projection_dim=1280,
58
+ torch_dtype=torch.float32,
59
+ )
60
+ tokenizer = CLIPTokenizer.from_pretrained("laion/CLIP-ViT-bigG-14-laion2B-39B-b160k")
61
+
62
+ prior = PriorTransformer.from_pretrained("ECLIPSE-Community/Lambda-ECLIPSE-Prior-v1.0")
63
+ pipe_prior = KandinskyPriorPipeline.from_pretrained(
64
+ "kandinsky-community/kandinsky-2-2-prior",
65
+ prior=prior,
66
+ text_encoder=text_encoder,
67
+ tokenizer=tokenizer,
68
+ ).to("cuda")
69
+
70
+ pipe = DiffusionPipeline.from_pretrained(
71
+ "kandinsky-community/kandinsky-2-2-decoder"
72
+ ).to("cuda")
73
+
74
+ raw_data = {
75
+ "prompt": args.prompt,
76
+ "subject_images": [args.subject1_path, args.subject2_path],
77
+ "subject_keywords": [args.subject1_name, args.subject2_name]
78
+ }
79
+ image_emb, negative_image_emb = pipe_prior(
80
+ raw_data=raw_data,
81
+ ).to_tuple()
82
+ image = pipe(
83
+ image_embeds=image_emb,
84
+ negative_image_embeds=negative_image_emb,
85
+ num_inference_steps=50,
86
+ guidance_scale=7.5,
87
+ ).images
88
+
89
+ image[0]
90
+ ```
91
 
92
  ## Important Notes (and limitations):
93