README.md

metadata

tags:
  - stable diffusion
  - animatediff
  - motion lora
language:
  - en

• Animatediff V2 motion LORAs
  • File summary
  • General recommendation
  • Examples
    • v2 Drone orbit right
    • v2 Drone orbit left
    • v2 Drone backing
    • v2 Drone forward
    • v2 Drone walking back-view
    • v2 walking front-view
• Animatediff V3 motion LORAs
  • File summary
  • General recommendations
  • Drone LORAs
  • Ballerina spin LORAs
  • Walking back-view LORAs

Animatediff V2 motion LORAs

File summary

Name	Compatible Motion Model	Resolution (w/h)	Frames	input videos
280_cseti_2906282_drone-orbitright-mv2_r64_w576_h384_fr16.safetensors	v2, animateLCM	576x384 (3:2)	16	1
280_cseti_6933991_drone-orbitleft-mv2_r64_w576_h384_fr16.safetensors	v2, animateLCM	576x384 (3:2)	16	1
280_cseti_6160305_drone-backing-mv2_r64_w576_h384_fr16.safetensors	v2, animateLCM	576x384 (3:2)	16	1
260_cseti_8890531_drone-forward-mv2_r64_w576_h384_fr16.safetensors	v2, animateLCM	576x384 (3:2)	16	1
200_cseti_3413444_drone-walking-mv2_r64_w576_h384_fr16.safetensors	v2, animateLCM	576x384 (3:2)	16	1
500_cseti_8315653_drone-orbitright-3vid_mv2_r64_w576_h384_fr16.safetensors	v2, animateLCM	576x384 (3:2)	16	3
600_cseti_4941259_walking-frontview-6vid_mv2_r64_w576_h384_fr16.safetensors	v2, animateLCM	576x384 (3:2)	16	6

General recommendation

• These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models
• For the drone LORAs, keyword "drone" or "drone shot" might help with the motion
• Animatediff v3 adapter lora is recommended regardless they are v2 models
• If you want more motion try incrasing the scale multival (e.g. to 1.2-1.5)
• I recommend using the above resolutions and upscale the animation or keep at least the aspect ratios
• The drone LORAs are trained on landscape footage so might don't work very well on portraits however can give you interesting results

Examples

v2 Drone orbit right

v2 Drone orbit left

v2 Drone backing

v2 Drone forward

v2 Drone walking back-view

• recommended to use words like "back-view", "from behind" in your prompts

v2 walking front-view

• recommended to use words like "front-view"

Animatediff V3 motion LORAs

File summary

Name	Compatible Motion Model	Resolution (w/h)	Frames	Comments
250_cseti_Drone-Rotating_3D_footage_fr16	v3	384x384 (1:1)	16	Drone 3d rotation
400_cseti_9222750_drone-rot_w576_h384_fr32_r64	v3	576x384 (3:2)	32	Drone 3d rotation
200_3723723_cseti_ballerina-spin_w512_h768_r64	v3	512x768 (2:3)	16	Ballerina spin
250_3723723_cseti_ballerina-spin_w512_h768_r64	v3	512x768 (2:3)	16	Ballerina spin
400_cseti_5294370_walk_backview-1_w576_h384_fr32_r56	v3	576x384 (3:2)	32	Walking back view
450_cseti_5294370_walk_backview-1_w576_h384_fr32_r56	v3	576x384 (3:2)	32	Walking back view

General recommendations

• These are finetuned on v3 model. Does not work with v2 or animateLCM motion models
• Animatediff v3 adapter lora is recommended
• If you want more motion try incrasing the scale multival (e.g. to 1.2)
• I recommend using the above resolutions and upscale the animation or keep at least the aspect ratios

Drone LORAs

• For more motion, try incrasing the LORA strength or try adding the below trigger words to your prompt
• Try trigger words: dronearound, flying around

Ballerina spin LORAs

• Sometimes a white veil / skirt appears during rotation. Unfortunately, this is from the original video. In such cases, try a new generation or reduce the LORA strength.
• I recommend using LORA strength of 0.8-0.9

Walking back-view LORAs

• I recommend using LORA strength of 0.8-0.9
• Recommended prompts:
  • "back view"
  • "drone shot" - for an upper view