|
|
|
--- |
|
license: creativeml-openrail-m |
|
base_model: SG161222/Realistic_Vision_V4.0 |
|
datasets: |
|
- recastai/LAION-art-EN-improved-captions |
|
tags: |
|
- stable-diffusion |
|
- stable-diffusion-diffusers |
|
- text-to-image |
|
- diffusers |
|
inference: true |
|
--- |
|
|
|
# Text-to-image Distillation |
|
|
|
This pipeline was distilled from **SG161222/Realistic_Vision_V4.0** on a Subset of **recastai/LAION-art-EN-improved-captions** dataset. Below are some example images generated with the finetuned pipeline. |
|
|
|
 |
|
|
|
|
|
This Pipeline is based upon [the paper](https://arxiv.org/pdf/2305.15798.pdf). Training Code can be found [here](https://github.com/segmind/distill-sd). |
|
|
|
## Pipeline usage |
|
|
|
You can use the pipeline like so: |
|
|
|
```python |
|
from diffusers import DiffusionPipeline |
|
import torch |
|
|
|
import torch |
|
from diffusers import DiffusionPipeline |
|
from diffusers import DPMSolverMultistepScheduler |
|
from torch import Generator |
|
|
|
|
|
path = 'segmind/portrait-finetuned' # Path to the appropriate model-type |
|
# Insert your prompt below. |
|
prompt = "Faceshot Portrait of pretty young (18-year-old) Caucasian wearing a high neck sweater, (masterpiece, extremely detailed skin, photorealistic, heavy shadow, dramatic and cinematic lighting, key light, fill light), sharp focus, BREAK epicrealism" |
|
# Insert negative prompt below. We recommend using this negative prompt for best results. |
|
negative_prompt = "(deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime:1.4), text, close up, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck" |
|
|
|
torch.set_grad_enabled(False) |
|
torch.backends.cudnn.benchmark = True |
|
|
|
# Below code will run on gpu, please pass cpu everywhere as the device and set 'dtype' to torch.float32 for cpu inference. |
|
with torch.inference_mode(): |
|
gen = Generator("cuda") |
|
gen.manual_seed(1674753452) |
|
pipe = DiffusionPipeline.from_pretrained(path, torch_dtype=torch.float16, safety_checker=None, requires_safety_checker=False) |
|
pipe.to('cuda') |
|
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) |
|
pipe.unet.to(device='cuda', dtype=torch.float16, memory_format=torch.channels_last) |
|
|
|
img = pipe(prompt=prompt,negative_prompt=negative_prompt, width=512, height=512, num_inference_steps=25, guidance_scale = 7, num_images_per_prompt=1, generator = gen).images[0] |
|
img.save("image.png") |
|
``` |
|
|
|
## Training info |
|
|
|
These are the key hyperparameters used during training: |
|
|
|
* Steps: 131000 |
|
* Learning rate: 1e-4 |
|
* Batch size: 32 |
|
* Gradient accumulation steps: 4 |
|
* Image resolution: 768 |
|
* Dataset size - 7k images |
|
* Mixed-precision: fp16 |
|
|
|
|
