svjack commited on
Commit
cd278b8
·
verified ·
1 Parent(s): 17a707e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md CHANGED
@@ -82,3 +82,57 @@ display.Video("char_scene_mix_{}.mp4".format(i))
82
  - With lora
83
 
84
  <video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/634dffc49b777beec3bc6448/kJ1t44JEkMuzUTukytX_K.mp4"></video>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
  - With lora
83
 
84
  <video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/634dffc49b777beec3bc6448/kJ1t44JEkMuzUTukytX_K.mp4"></video>
85
+
86
+ # STG Example (use STG make better video)
87
+ # Additional Installtion
88
+ ```bash
89
+ git clone https://github.com/svjack/STGuidance
90
+ cd STGuidance/diffusers/mochi
91
+ ```
92
+
93
+ ## LandScape Example
94
+ ```python
95
+ import torch
96
+ from pipeline_stg_mochi import MochiSTGPipeline
97
+ from diffusers.utils import export_to_video
98
+ import os
99
+
100
+ # Load the pipeline
101
+ pipe = MochiSTGPipeline.from_pretrained("genmo/mochi-1-preview", variant="bf16", torch_dtype=torch.bfloat16)
102
+
103
+ pipe.load_lora_weights("svjack/mochi_game_mix_early_lora")
104
+ pipe.enable_model_cpu_offload()
105
+ pipe.enable_sequential_cpu_offload()
106
+ pipe.vae.enable_slicing()
107
+ pipe.vae.enable_tiling()
108
+ pipe.enable_vae_tiling()
109
+ #pipe = pipe.to("cuda")
110
+
111
+ #--------Option--------#
112
+ prompt = "The video presents a tranquil scene of a small, isolated island with a rocky outcrop. The island is covered in lush greenery and dotted with vibrant pink cherry blossom trees in full bloom. A traditional-style building with a pagoda-like roof stands prominently on the highest point of the island, suggesting a cultural or historical significance. The sky above is a gradient of soft pastel colors, transitioning from light blue to pink, indicating either dawn or dusk. The water surrounding the island is calm, reflecting the colors of the sky and the island's features. There are no visible people or moving objects, giving the scene a serene and untouched quality."
113
+ stg_mode = "STG-R"
114
+ stg_applied_layers_idx = [35]
115
+ stg_scale = 0.8 # 0.0 for CFG (default)
116
+ do_rescaling = True # False (default)
117
+ #----------------------#
118
+
119
+ # Generate video frames
120
+ frames = pipe(
121
+ prompt,
122
+ num_frames=84,
123
+ stg_mode=stg_mode,
124
+ stg_applied_layers_idx=stg_applied_layers_idx,
125
+ stg_scale=stg_scale,
126
+ do_rescaling=do_rescaling
127
+ ).frames[0]
128
+
129
+ export_to_video(frames, "Island_scene_mix_stg.mp4")
130
+ from IPython import display
131
+ display.clear_output(wait = True)
132
+ display.Video("Island_scene_mix_stg.mp4")
133
+ ```
134
+
135
+ - With lora and STG
136
+
137
+ <video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/634dffc49b777beec3bc6448/vuYzwwwun99L9LEB7HHqP.mp4"></video>
138
+