|
--- |
|
license: mit |
|
tags: |
|
- stable diffusion |
|
- animatediff |
|
- motion lora |
|
language: |
|
- en |
|
--- |
|
- [Basic camera motion LoRAs for Animatediff v2 motion models - version 1](#basic-camera-motion-loras-for-animatediff-v2-motion-models---version-1) |
|
- [Model Description](#model-description) |
|
- [File summary](#file-summary) |
|
- [How to use](#how-to-use) |
|
- [General recommendation](#general-recommendation) |
|
- [Details and examples](#details-and-examples) |
|
- [Zooming in (16 frame)](#zooming-in-16-frame) |
|
- [Zooming in (32 frame)](#zooming-in-32-frame) |
|
- [Zooming out](#zooming-out) |
|
- [Lateral (horizontal) left](#lateral-horizontal-left) |
|
- [Lateral (horizontal) right](#lateral-horizontal-right) |
|
- [Crane (vertical) up](#crane-vertical-up) |
|
- [Crane (vertical) down](#crane-vertical-down) |
|
|
|
# Basic camera motion LoRAs for Animatediff v2 motion models - version 1 |
|
|
|
## Model Description |
|
This repository contains a collection of basic camera motion LoRAs finetuned on Animatediff v2 models. The training process utilized Kijai's [AnimateDiff Motion Director ComfyUI nodes](https://github.com/kijai/ComfyUI-ADMotionDirector). These LoRAs have been tested exclusively in ComfyUI with Kosinkadink's [AnimateDiff-Evolved nodes](https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved), however they may be compatible with other AnimateDiff implementations. |
|
|
|
## File summary |
|
|
|
| Name | Compatible Motion Model | Base AR (w/h) | Frames | input videos | |
|
| ------------------------------------------------------------- | ----------------------- | ------------- | ------ | ------------ | |
|
| 1600_cseti_9192119_camera-zoomout-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
|
| 1600_cseti_1077723_camera-zoomin-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
|
| 2800_cseti_1093371_camera-zoomin-32f-10vid_mv2.safetensors | v2 | (3:2) | 32 | 10 | |
|
| 1600_cseti_6664682_camera-lateral_right-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
|
| 1600_cseti_9213319_camera-lateral_left-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
|
| 1600_cseti_9406077_camera-crane_down-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
|
| 1600_cseti_1720230_camera-crane_up-10vid_mv2.safetensors | v2 | (3:2) | 16 | 10 | |
|
|
|
## How to use |
|
- copy the motion lora files into your "ComfyUI\models\animatediff_motion_lora" folder |
|
- Use the "Load Animatediff LoRA" node |
|
|
|
## General recommendation |
|
- These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models |
|
- They work with SD 1.5 checkpoints |
|
- Animatediff v3 adapter LoRA is recommended regardless they are v2 models |
|
- If you want more motion try increasing the scale multival |
|
- I recommend using the above Base AR (Aspect ratio) for inference however they might work with other ARs too |
|
- Try playing with the strength of |
|
- motion LoRA |
|
- scale multival |
|
- v3_adapter lora |
|
|
|
like increasing the scale multival and lowering the lora strength. |
|
- I recommend using IP-Adapter, however it reduces the effect of the mLORAs, so play with the strength. "Ease in out" weight type is a good starting point. |
|
- If you chain multiple motion LoRAs, I recommend lowering their strength, otherwise, the quality of the result will be worse |
|
|
|
## Details and examples |
|
|
|
### Zooming in (16 frame) |
|
- recommended strength: 0.7-1.0 |
|
|
|
<video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-zoomin16.mp4" title="zoomin 16 frames"></video> |
|
|
|
### Zooming in (32 frame) |
|
- this is an experimental motion LoRA trained on 32 frames so it can be used with context length 32. |
|
- recommended strength: 0.7-0.85 |
|
|
|
<video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-zoomin32.mp4" title="zoom in 32 frames"></video> |
|
|
|
### Zooming out |
|
- recommended strength: 0.7-1.0 |
|
|
|
<video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-zoomout16.mp4" title="zoom out"></video> |
|
|
|
### Lateral (horizontal) left |
|
- recommended strength: 0.7-1.0 |
|
|
|
<video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-lateral_left.mp4" title="right view"></video> |
|
|
|
### Lateral (horizontal) right |
|
- recommended strength: 0.7-1.0 |
|
|
|
<video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-lateral_right.mp4" title="right view"></video> |
|
|
|
### Crane (vertical) up |
|
- recommended strength: 0.7-1.0 |
|
|
|
<video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-crane_up.mp4" title="right view"></video> |
|
|
|
### Crane (vertical) down |
|
- recommended strength: 0.7-1.0 |
|
|
|
<video controls src="https://huggingface.co/Cseti/Basic_camera_motion_LoRAs_sd15-ad2-v1/resolve/main/asset/LoRA-comparison-crane_down.mp4" title="right view"></video> |
|
|
|
|
