Motion Extraction
HTML-код
- Опубликовано: 2 окт 2024
- With a very simple trick you can extract the motion of a video. All footage was shot by me, as usual...
Music*: posy.bandcamp....
When requested a lot I'll add a single of the end song.
Support my projects on Patreon: / posy
Tutorial attempt: • Motion Extraction tuto...
Posy on Spotify: open.spotify.c...
Or on Apple Music: / posy
![FendiDa Rappa - Clock Dat ft. Shamar Marco [Official Music Video]](http://i.ytimg.com/vi/3NBU8KHdxYY/mqdefault.jpg)
This video is both a tutorial, a demonstration, and a work of art. Bravo.
Tutorial? Really?
No math, no code, no explanation on computing. Just nice video - nothing else
my thoughts exactly
agreed
@@TushhsuT he explains how to do the effect in the video... 1:15
@@arcadeprism I don't follow the explanation because I don't know what he means by "invert" can any one help me out.
You could've made this a plug in and charged €89 for it, but you just told us how to do it for nothing. Fantastic work as always.
there’s a company that charges several orders of magnitude more for a service that looks very similar to this
I mean, if the concept is known, then technical side isn't really that hard, someone would do the same thing but cheaper, eventually someone would make something like this but open source
The concept is absolutely simple and one of the most basic effects, subtract one image from another. This is what the inversion at half opacity technically does.
I think similar technology used in motion amplification cameras, and yeah, they cost a fortune, but you basically can do the same with regular video editing software
@@llMarvelousim guessing those are also high speed cameras, which justifies a higher price range a bit. High speed cameras are marvels of engineering, to me anyway.
'And just like that, seven minutes go by.' Those effects were mesmerizing. What a great seven minutes indeed.
WWWWWWWWWWOWOOWOWOWWOOOOOOOOOOWOWOWOWOWWWWWWWW :OOOOOOOOO
I smiled for that 7 minutes and more
It's strangely keeping me watching too... Never knew seeing movement only, could be so pretty...
I was left in utter shock watching this video, totally speechless but then 6:55 happened and it brought me back
Same LOL
WWWWWWWWWWOWOOWOWOWWOOOOOOOOOOWOWOWOWOWWWWWWWW :OOOOOOOOO
I DIDNT EVEN NOTICE THAT
Guy talking about his latest breakthrough in a serene voice while showing you the impressive results is my favorite genre of video
What's this genre called? Bc I like it as well and want to find more clips like this
I think your editing and your narration on top of this “motion extraction” is what made this video really great.
So what made the video great was… the video?
@bacherfkinmcskiddlywop2491yes
@@3RR0RNULLpretty much
The narration was super annoying
@@48956l no... and he even said sorry at the end!
As a compositor I have used motion extraction often for separating parts of an image or removing snow/rain. I never really tried to use it as it's own effect in such an artistic way. So cool!
how do you remove rain with motion extraction :o
Would you use it to add FX such as rain/smoke/distortion to an existing video?
I just thought about using this to make a mask to extract elements from live action plates. Particularly things that are almost impossible to roto like smoke or semi-transparent cloth.
I suppose you can just use a difference map if you've got a static camera locked off shot, where the element you want to roto comes in after the shot starts, but this also works even if you never get a single clean plate of your background.
@@Somerandom1922 you can also just stabilized that part of the image with a planar track if you have camera movement and use motion to extract the rest
@@staymaddie rain usually exists as bright streaks or dots on the video, so by duplicating the video on top of itself and time-shifting it (possibly multiple times), and using a blend mode that only keeps the darkest version of the pixels in a given area (e.g. min/darken), you can effectively remove rain from a stable or stabilized plate. when dealing with stabilized footage: if you did the stabilizing manually (if you have the values of the repositioning), you can invert the data, and put it on the rain-free version, getting the original camera movement back, but now without rain. Yes, i am a compositor as well :)
Nicely done! Thanks for sharing how to do the effect. There's a paper from some years ago that used motion analyzed footage from high frame rate cameras to extract audio from the video of resonating objects. Seems like this method might simplify DIYing that process. There's also software that amplifies motion to detect unwanted vibrations and loose connections in machinery. That would be really useful for home auto and appliance repair. This might get the job done for that purpose as is. Maybe you could sweep through camera shutter speed in the footage to detect particular vibration frequencies, doubling this method with a stroboscopic effect.
Cant wait to see that video 😊
I saw a Steve Mould video where he talked about it.
I was also reminded of the vibration detection software - I think this looks cooler. As for somehow mixing it with stroboscopics, well, I might just buy my own camera if it's as cool as it sounds.
Fireworks might look cool with this too... gonna try making it myself
I remember that Ted talk! I always thought it sounded kinda suspicious but this video has certainly convinced me.
My mind immediately went to home surveilence/hunting cameras. For hunting, write a script to find a path of detected motion of a minimum pixel area (this limit perhaps modified by the Y position if the camera is observing a wide shot). Similar for home surveilence. Either of these could then trigger high intensity scrutiny via AI object detection. Another novel use would be with the chroma channel shifting he showed later... an augmented reality live feed from an external camera would be a very aesthetic way to have a security feed visible in a public space.
~PS: love your content
I laughed out loud when you said, "And just like that, 7 minutes go by!"
This is an amazing contribution!
I laughed out loud when Posy cut cheeze @6:55
As an amateur video editor I really appreciated this video. Not only did this unlock a whole new world of creativity, it was also quite beautiful. Thank you.
I am also quite armature but this is crazy. Also Who do you think would win in a fight: John Wick or Harry Hart?
This should definitely be one of your top videos, it could inspire so many art projects
I'm wondering if this would be useful in forensics. To make it easier for people to spot things in footage which they might not have otherwise.
its used in factories to spot if screws are loose or if equipment is resonating@@zemja
Nothing beats his VFD videos 😇
i could imagine this being in some sort of surreal animation project
I read this comment just as Posy played a fart noise at the end.
You just blew my mind,
I'm a graphic designer and I worked on video footage a lot, even composing abstract video clip for music artists and other projects. I've never seen something like this.
This will for sure inspire me for the future compositing and video projects, thank you !
That sounds amazing! Good luck!!
Yeah, this actually seems like it could be great for some sort of animation style as well.
Its absolutely criminal that you don't have more subscribers. This was sublime.
he has quite a few
@@zanaisuAnd he needs more
Agreed! This video just earned my subscription.
Why do people always say this stupid line? Some dude could litearlly post their first video, it is amazing, and an oaf like you will be like "Why don't you have more subs?!??!?"
And he has 295k, which is a decent amount, so I don't even know what you are talking about.
@@pyropulseIXXI well, thank you for calling me an oaf I guess. That's a cool thing that happened today.
Videos like this will help so many people have a wider perception towards how not only is video the medium most art is made through but it can also show us details we can’t see ourselves
*"And ofcourse you could also use color, if you're into that kinda thing"*
05:48 * Blows my mind
I always loved posy's videos. They're just masterpieces.
every video is like a damn fine art masterpiece and it's always worth the wait.
omg. it's the real egg. no, like, for real.
His movies are education art works.
I loved them as well. I still do too
It's the real youtube premium.
I did _exactly_ this about 25 years ago ! This was before progressive footage was widespread, back then I was looking for a way to de-interlace interlaced (50i PAL) footage, but I didn't want to lose too much vertical resolution, so I ended up extracting the motion, using that as a matte so I could de-interlace the moving parts of the image (where interlacing showed the most) and leave the more static areas (where interlacing isn't visible) at full resolution, it worked pretty well !
Wow! That's so cool!
I also made a similar filter several years ago but did not upload the results (yet). But Posy's results are really beautiful I must admit!
that's pretty much how modern "smart" deinterlacing algorithms work
I'm so glad interlaced footage has gone away for the most part.
WWWWWWWWWWOWOOWOWOWWOOOOOOOOOOWOWOWOWOWWWWWWWW :OOOOOOOOO
The fact that you don’t use stock footage in these videos is simply mid blowing you most definitely should have more subscribers
WWWWWWWWWWOWOOWOWOWWOOOOOOOOOOWOWOWOWOWWWWWWWW :OOOOOOOOO
This channel started out as a music video channel where he used photography/landscape videography as a backdrop for his music. The whole thing pleasantly morphed into what it is today organically--he didn't start out trying to make educational or even narrated videos, but when he moved to this format he took all the gorgeous, carefully made works of photographic art with him
I was already enthralled by the effect but the aplication at 2:52 completely blew my mind. You're truly one of the most ingenious creators in this platform. Love to see your work
Came here on Benn Jordan's video recommendation. Glad I did. Beautiful. 👍
the windmill shot at 5:41 blew me away. incredible work, and thank you for sharing!
That and the ferns one are art, they would be great looping GIFs
That shot convinced me I had a dangerously high fever and had just had "special" mushrooms.
I want to recreate it because it’s so pretty
Excellent! This reminds me of the "motion amplification" process, that is used to survey the mechanical vibrations. By changing the framerate of the recorded footage, the surveyor can tune in on the resonant frequency of the device/structure, and then the motion amplification visually amplifies the movement, usually completely imperceptible by a naked eye. This allows to see the wobble that will in time damage the components, or even whole buildings - and prevent that by highlighting the areas that need attention/fixing.
I was just about to bring that up myself :D
Perhaps a video from Steve Mould?
A lot more goes into motion amplification than changing the frame rate. They do some frequency domain boosting.
So amazing how you combine art, science and playfulness, and something enjoyable comes out! Keep up the good work! :)
I was subscribed for a while for the audio stuff, but this is a hidden gem I just found.
This is the third time I've watched this and i still am amazed, thank you and i have to say you are amazing!
You took a very simple thing, that a lot of editors have probably played with at one point or another, but then went to the moon and back with it. Absolutely love the results.
It's just a difference mat and used all the time for all kinds of things.
A boring approach is used for comments like that all the time, just different opinion about other people's learning. I think calling things "just a common quality" is a common quality of comments that approaches boredom for me. This is just a difference sentence ;) Learn on everyone @@oBCHANo
I have only one thing to say: This is art.
Thank you for making these videos. They look so good, you could probably work for a TV manufacturer to make demo footage.
This is the only channel that can bring back that childlike wonder and amazement Vsauce videos used to make me feel as a kid
o:
"...as a kid."
siiiighhhh.... and now I feel old.
thank you ghettosmosh
Lol you're young. (alternative comment: lol I'm old)
celebrities shop too...
This is one of the most beautiful tools Ive ever seen
This is the definition of visual ART! thank you for teaching us how to do it as well!
i feel pity for all the people who still didn't discover this amazing gem of a channel.
I said it before and I'll say it again - The most mesmerising channel on YT! The knowledge and versatility forged into bitesize chunks which will inspire and encourage you to try and look differently on things around. This process was impossible to replicate from Steve Mould's video, Posy on the other hand - 4 steps! Duplicate, Invert, Shift, Change opacity to 50% - as simple as that. And the quality of his footage - absolutely new level. I want to see this on trending as I saw the LCD Display video which made me watch every content of his in one go. Absolute legend!
This channel is a hidden gem of youtube.
Yeah I know right, I find everything posted here extremely enthralling.
Whoa. This one goes straight into my favourites.
This is incredible but I am still lost a little at how you isolate the motion and colors so well. Could you show how you do this in more detail? I want to use this in “haunted” locations
You basically have multiple videos half-see-through, like looking at a picture through a picture
You then shift the colors of each one, and make it so there is a delay on one of them
The parts that arent moving are the same in the time 15:45 in a video and in 15:46 but the parts that are moving will be changed, this means any motion will desynchronise the colors between the videos making those results
@@eeeguba432 I understand the basic concept, but when I did all that it does not look as good as he demonstrated in the video. I am requesting a video with more detailed steps. Thanks though.
@@carlcrusher ahhh, thats what you meant, whoops
Well, i guess if someone is more confused they will have an explanation there lol
He has a tutorial on his second channel if you're still looking for more info.
Try using the "difference" blend mode on the second layer while keeping it 100% opacity and positive colors.
You can then add another "original" layer beneath these two. If you want to have the movement over the original layer, and then select "lightness" as a blend mode for the middle layer. It would work better maybe if you export the movement layer beforehand. But might still work. Haven't tried his "inverted + 50% opacity" solution though.
I've said it before but you are genuinely one of if not the best creator on RUclips. No other creator gives me such big feelings the way your visuals and music have. You are a massive inspiration. I appreciate your hyperfocus on the little things and it makes me feel a little less alone. Thank you for sharing this stuff with us
Truly fascinating. My jaw dropped when you started shifting the color channels! This is such a well made video. The shots, compositions, effects, music, narration, the creativity. Incredible. Thank you for making this!
This is actually really cool for making particle effects, for example with the fireflies. You can overlay it on something else with a additive effect
This is beautiful. The way you captured seemingly invisible motion is incredible
Thanks Benn Jordan for linking this video it was worth it
I can't believe I never thought to do this! Those black backgrounds on particles like pollen, bugs, rain, snow, ect. are perfect for compositing, it's insane to me that I never thought to do something like this to extract those particles. I do it the hard way and simulate them with CG!
this is one of the most fascinating effects I've ever seen and I can't believe I can just replicate it myself for free because you told it to me rather than sold it to me
Some videos shouldn't stop. Brilliant subject with quite intriguing visuals as always! Awesome!
Your RUclips content is on a different dimension to everyone elses. Incredible
sthis just blew my mind, and made me think of so many ways to make effects now, this is insane
Awesome demonstration! That principle is used in most video coding algorithms and called DPCM (in interframe codecs). So only the moving parts of the image have to be encoded while the still parts are only encoded in certain intervals. Together with motion prediction it saves a ton of bandwidth at the cost of higher codec complexity and latency. H.246, MPEG4, HEVC or VP9 are just a few examples here.
When reading about how video codecs work, I had always wondered "but how do you track motion?" It seemed so complicated to analyze objects and track how they moved, vs. just new or even existing visual detail. And then I see this, and it gets 100x easier to see how it would be done.
Which seems really obvious now, as inversion and summing are essential building blocks of audio manipulation (M/S, surround decoding, glitch removal, changes in phase -> EQ, etc.)
06:04 Aurora Borealis? At this time of year! At this time of day! In this part of the country! Localized entirely within that clearing?
Math explanation! You have managed to calculate the approximate time derivative of your video in one efficient step! Your video is a signal f(t), adding the inverse to it gets you f(t) - f(t), then shifting the additional part gives you f(t) - f(t + dt) for a small time change dt. This expression is proportional to the first derivative in time, which corresponds to first-order change, i.e. things that have a "velocity" in the video. By changing the time shift, you are effectively looking at different scaling of the first derivative. Really great stuff.
This is truly one of my most favourite videos on RUclips and I love to come back from time to time to enjoy this pearl
I can't even describe what your videos do to my brain but it feels cosy and satisfying. Your childish humour and all your content is so refreshing. Reminds me of the wonders of exploring the world through children's eyes. Cosy-Posy
you just perfectly captured my last acid trip. Amazing Video
This is brilliant! I was totally engrossed, couldn’t believe when you said 7 minutes had passed by!
This is very inspiring art! it's so simple but so versitile and you can probably make so much more stuff with it! Maybe a music video, maybe a movie, maybe just a compilation of cool clips! I'm really impressed that you keep finding these seemingly simple things and turn them into mindblowing experiences!
the photography, editing, and sound design are beyond incredible
Can't even begin to describe what this video made feel.... It's just amazing.... We are nothing but a constant state of change, and I find that so beautiful.
you can also just use the difference blend mode at 100% opacity. I use this all the time, it's great for lining things up in space and time
This effect is awesome, and some of those shots are breathtaking even before it was applied. The quality of your videos continues to amaze me.
I feel like the magical energy of the world has been captured. This is so calming and beautiful
Uhhhhh new skillset unlocked. I'm totally going to start making movies or offers shoots with this. This is a WHOLE new playground to push the limits with. Thank you so much for showing us this!
This was so mesmerising and peaceful. I loved your speaking tone.
Posy I'm so grateful that you document the destinations that your curiosity takes you. I've never seen better use made of a good video camera. Fantastic work as always 🎉
Benn Jordan brought me here.... And I am going to try that.
00:16 I call this, df/dt
Differential equations where hard for me to grasp. Because it's a slope field, harder to do on calculators. I concluded differential equations by just using Laplace transforms, not primary differential techniques.
This actually really useful to check if a machine have a problem, like something moving when they shouldn't. VERY COOL EFFECT! Nicely done
This is one of the most beautiful videos I've ever seen. The video, the music, the voice... and it's even educational. Masterpiece!
In addition to motion amplification software others have mentioned, these images you made are very similar to new type of camera that's being developed called "event cameras", "dynamic vision sensors", or "neuromorphic cameras". Their purpose is to only record motion by tracking the changes in the image (i.e. change in light for each pixel receives on the camera sensor). I think you've just replaced a lot of the usefulness of these new cameras being developed, although they can record at much higher frame rates. The processing methods you described could be applied easily by digital image signal processors in modern cameras today to make near real-time video recordings for people only interested in extracting motion information. There are many many applications for event motion extraction. If this method you show in your video hasn't formally been described before, I think it's worthy of a scientific paper.
Absolutely agreed.
This is so cool! It reminds me of the motion amplification effect Steve Mould did a video about a while ago (I presume that involves lower level manipulation - presumably multiplying the values in I-frames by some factor or something).
I don't really have any cause to do video editing, but this makes me want to dig through old videos and see if there's anything I can apply it to. Very inspiring.
It reminded me of the same thing!
What Steve Mould did a video on was more about motion amplification and less about simply highlighting motion like in this video. I'd imagine the first step to motion amplification is something similar, but it's definitely a more complicated process than this. They're both really cool and accomplish very similar things though.
@@tbuk8350 Yes; this is essentially filtering information from the time domain, whereas motion amplification actually modifies information in the time domain (i.e. increasing the distance a pixel or macro block travels in a frame). As you say, that's more involved; as I said above, my suspicion is that it takes advantage of the structure of video compression formats. Namely, intermediate frames between full stills only store the transformations that the computer has to apply to transform those stills into the intermediate frame. Those instructions could presumably be modified to increase those changes, be they linear motion, rotation, or change in colour. But that's not something that video editing software offers (why would it?), so it would involve quite technical low-level knowledge of video formats, etc.
this is amazing yet so simple to do. edit: make sure to use linear light blend mode if in premiere pro
W comment
Excellent music - I've seen this sort of thing before, but I never really appreciated how beautiful it was
Benn brought me here, but I think I'll stick around! Great video, Posy!
0:53 wait… couldnt you key with this technique? Or am I missing something? You could Export the Video and make it a mask no?
I don't know much at all about video production, cameras, photography, or editing but in 7 minutes, you've made my brain create hundreds of ideas. Bravo, an excellent piece of art and an informative video.
well, go out there and give it a try! im sure you might enjoy it at least
@@cozz124 man thanks for reminding me about this video! I think it helps that I'm a creative, just in a slightly different field - music.
I unfortunately don't have much time to dedicate to this amazing artform in my life right now but hopefully in a few months I might!
Always great to see a video notification from Posy. Imma go now and get some snacks to eat while enjoying this video
edit: Just watched the video and damn, such an interesting concept. It's so easy to achievable and looks very pretty too. I would love to see more videos like this lol
did you bring enough for the whole comment section?
I remember watching a video on motion extraction through code, but I didn't even consider literally just using a video editor.
I can also imagine this is a perfect way to extract temporal noise maps. This is incredible. Thank you.
you made my day just that little bit better, man. thank you for sharing this lovely poetic piece of craft!
5:58 im searously wondering if this shift in colors, in time, differentilaly can account for some of the fenomenology of LSD and other psychedelic drugs!
Fantastic video! The technique you've shown for isolating the moving parts in a video has several practical applications, including in the field of video compression. In many compression algorithms, the focus is on capturing the differences between consecutive frames. This approach ensures that only the changing elements in each frame are stored, leading to more efficient data usage. What's particularly fascinating is the evolution of these algorithms. While early methods relied on basic difference extraction, modern algorithms employ sophisticated prediction techniques. These advanced methods not only identify changes but also predict them, leading to significantly more efficient compression. This evolution marks a remarkable progression in our ability to handle video data more effectively. Great work demonstrating this concept!
Yeah, the example where he first shows how to do it and the motion appears, it looks exactly like a playing a video file with corrupted keyframes
It is actually used for industrial equipment to see if everything is in phase and not moving to much. I was mesmerised but thought it would be much more complicated. 🎉 Thanks!
we need a full tutorial on this, incredible
I've got a github repository with a python implementation if you want. Not sure i can link it here
@@K4leidosif you’re serious maybe you could give it a try and paste a link - after all links do work in the video notes? Or maybe just give us the name of the repo? (a quick search on your name didn’t find it, but if you give your GitHub username and repo title that would work..
@@K4leidos Could you please tell your github username? I'm very interested in trying!
You are taking the derivative of the video. This is very cool. I feel like I need to show this to my calculus students the next time I teach it.
You are one unique artist... I really really love how you see the world.
Is the ending music on Spotify? 6:18
Posy your videos are always so incredible, thought provoking, and usually also a visual spectacle; thank you for what you do.
Absolutely amazing.
The chromatic shifting is so amazingly beautiful through the simplist means. Makes me think of rainbows.
You truly are an artist. I could just watch this on and on. The endless visual soundtrack to life. And it ends just like life often does. Exceptional observational skills.
the way you made an analogy between the visuals and the music at multiple points was impressive
This is amazing! Holy cow, it scratched itches in my brain that I didn't even know about! That was some amazing complex use of simple tools and I am blown away! Thank you! Thank you! The shots of the rain droplet ripples gave me goose bumps!
The music and foley is what did it for me 😻
Desperately need the ending song as a single on Spotify
I have never once been interested in photography or videography but this is one of the most interesting things I have seen in months. The ability to spot that deer is really what hit me, it's application can be both useful in that way, or beautiful if used like the rain drops one nearer the beginning
Woop woop that's the sound of da military industrial complex. WOOP WOOP
Very creative and beautiful video effects! I noticed that you can see this effect with your own eyes if you patiently sit very still and don't move your eyes for about 5 minutes. Then most of the visual field goes grey, except for moving things. Try it! I discovered it years ago while sitting on a hilltop and staring at a distant object, when I noticed I could see every ant moving in the dirt near me in my peripheral vision. (You have to resist the urge to move your eyes at that moment and just move the focus of your attention.) I would guess that some raptors use this to find rodents moving in the fields below.
Yes! I often did this while I was in school
Another way to bring out the motion is to try and generate motion vectors from the footage. They are harder to control visually, but could produce interesting patterns that might be useful.
Wow! This is awesome :) sent this to my little brother who really likes working with cameras, maybe itll give him some cool ideas 👍. Thank you for sharing this!
Potentially the best video on youtube.
The current thumbnail doesn't really attract viewers (I almost didn't click on it). I think you could get more views if you made a thumbnail with a split down the middle. You show a picture of the normal footage on the left side and show a picture of that same footage but with the motion extracted on the right side with a small arrow in between them pointing from left to right. You best pick one of the more visually striking ones that appear after 5:18
agreed i definitely was going to skip over it, also didnt get the title either. glad i didn’t
2:48 the footprint one seems right out of a spy movie
Your videos are always so calming and informative, you deserve recognition for your fantastic work
It's insane, it could have interesting applications if a program could do this live: hunting, tracking, etc..
It reminds me of this one video about motion amplifying cameras
i wonder if security cameras use this. spotting a deer where a camera normally can't, or foot steps in rocks? incredible surveillance use.
and the artistic uses are equally incredible. thank you for sharing this 😁
The RGB split just unlocked a whole new kind of effects for music videos.
Finding this absolutly incredible channel was one of the best things i have ever done, the professionalism behind this videos is just wow 😦