[Mostly Guernika] Relight image / Please preprocessors action
Hello
@GuiyeC
and friends !
It’s a weird request, but since you are my favorite mac dev…
I’ve been playing with GLSL shaders to imitate clipdrop.co/relight for animation.
I got something working so far, and I want to extend it.
I pushed it in Apple Motion / FinalCutPro via a 3rd party app (ISF Editor, ISF for Motion) that allows shaders, but it’s clunky, and a bit slow at realtime, probably because GLSL is deprecated.
It’s just a sunday experiment, because « why not? » but i’m happy so far :
Although it can animate intensities, ambient, shininess etc NO other light properties than color/position are animated in this quick demonstration (Motion)
(source : Pexels video)
Since it uses pre-generated Masks, Depth maps, Normal maps, I wish there could be an interface or a Shortcut action that generate those on the fly.
Or Even better, a direct plugin hooked to FinalCutPro/Motion but that’s for another day.
I can code, I can adapt, but Swift documentation and xCode freaks me out, I never learned it, so I'll be searching for basic swift, glsl to metal, then how to make plugins for FCP etc.
Have you any guidance or boilerplate that could help me leverage coreML models to make those series of images in an efficient way ? Like a dropzone that batch-preprocess
I’d give you the GLSL shader, it’s not magic,
do what you want with it, make an relighted app w/e.
(credit my real name for my curriculum lol)
I just want this tool in my Mac/iOS workflow, the existing tools are mostly for Adobe products (realtime is bad), overpriced and such…
Any, ANY suggestions welcome.
@ZProphete
You can use rembg to generate mask sequences in batches. The command is as follows:rembg p -w -om ./input ./output -m isnet-general-use
But I think this effect is very poor, and the mask of each frame will not smoothly.
I recommend you to use the powerful DaVinci Resolve with its own AI Mask, Relight, Depth Map, And the scope of these tools can be adjusted in real time.
That's exactly the point !
why using a command line ?
Resolve Studio 285€ is not worth relearning everything from AfterEffect, or I'm just lazy/demotivated.
As said, it's a 2hours sunday experiment, that only a few parameters were animated (that why it's poorly blended),
it could be educational and introduce me on swift, in a funny way.
Ended up using the official FxPlug documentation and got it to work in MTL, It's not thaaaaat bad.
Adding preprocessors to it is yet to learn !