I generated a photorealistic music video with AI (GauGAN2)
HTML-код
- Опубликовано: 1 мар 2022
- I fell in love with AI generated landscapes recently. Although this tool is only meant for images, I decided to see if making a cohesive video was possible. Choosing which parts of the landscape would represent different parts of the beat was a lot of fun! Maintaining continuity from one frame to another is challenging, but after much trial and error, I've found that minimizing the motion of the horizon maximizes cohesion between frames.
To support these projects and include your name in the next video, here is my Patreon page: / doodlechaos
Discord: / discord (I spoil what the next video will be pretty often here)
Twitter: / doodlechaos
Official tiktok: / doodlechaos
Shorts Channel: / @doodlechaosshorts2905
Instagram Shorts: / doodlechaos_shorts
Email: contact@doodlechaos.com
Here is a link to the web version if you'd like to try it yourself!
Here is a quick explanation of my process for this project:
1. Animate segmentation maps in sync with music in After Effects.
2. Export this animation as a series of images.
3. Convert the image sequence into landscape photos via GauGan2.
I was able to automate the conversion process. My first attempt used AHK to move my mouse and click to upload the correct image, convert, download, and name. This was sketchy and actually crashed my chrome browser somehow (I think due to downloading so many images and they were all being cached?). Attempt two was to run it locally. Unfortunately, the NVIDIA Canvas downloadable software doesn't allow png segmentation maps as input (only .can files). My final solution works by sending images directly to the gaugan.org server via python. If you're interested in trying out something like this, I'll make the script I used available for download on patreon.
Resources:
www.nvidia.com/en-us/research...
blogs.nvidia.com/blog/2022/03...
Music Credit:
Track: Lost Sky - Vision [NCS Release]
Music provided by NoCopyrightSounds.
Watch: • Lost Sky - Vision | Du...
Free Download / Stream: ncs.io/VisionID
Shoutout to Winkey for giving me the idea to make this Bob Ross themed. 
Развлечения
He's using his actual powers to terraform the Earth now
Who Bob Ross?😂
WOAH. That visualizer totally caught me off guard. Now that is amazing.
This is such an interesting medium. At first I thought it was a little weird but cool. But actually this is super cool.
@Anna ♪ Jeez, you gotta warn us before dropping something like that. I also don't see the relevance.
@@JoveRogers97 It's just a bot. Report and move on
@@oceanman207 I already did. My comment is not meant for Anna.
Its called acapella
Nice
That was super original. I love seeing experimental ideas like this!!!!!!
Thanks Creezy!
🔥👌
Calfreezy
@@DoodleChaosI noticed that the link doesn't work anymore, can you get a new one (although it may be because I'm on mobile)
Now that was different! I loved the part at 3:20!
Thanks! That part was the main reason for the epilepsy warning at the start.
ok
Me too! Best part
looked like an actual visualizer
@@Mereaux oh yeah yeah oh yeah
watch out guys, they'll be coming for you as an instrument next
I wouldn't even mind
@@SwarumtheForum in fact, that might be a good idea
Yea
What an interesting way to synchronise music!! Each frame is truly beautiful, I see how you fell in love with it! Nice job!! :D
I'm glad you like it! Thanks :D
yes
i've seen this AI but wow i've never seen it put like this :O
this is so cool, especially those sound waves at 3:14
Sup Groovy
@Jax Hin ai is getting smarter 😂
Npc getting more smarter than hoomans
@Jax Hin not npc, its ai
Plot twist: the npc made it
It's really cool how the picture moves with the music
the mountain moving remind me of my cat crawling under a blanket
Props to the AI for moving mountains, changing the terra and creating/drying ocean for this sick beat
It's crazy that an AI can be used to generate such realistic looking landscapes
This is such a creative idea. You even managed to seamlessly work your own logo in as an integral part of the video.
And the little behind-the-scenes at the end!
As someone who’s been following AI image generation for the past several years, it’s such a pleasure to be able to see it utilized to this degree. Very innovative and beautiful!
Awesome! Personally I love the part at 1:32. Every frame looks like a real landscape.
As someone who has played around with GauGAN a LOT, I commend you for your skills. It is truly incredible and hard to get consistent and looking how you want it. Well done.
What a clever idea! It's so cool that you could pause the video almost anywhere, and there would be a beautiful landscape. And the way the landscape flowed and changed was amazing!
The beauty of nature + the beauty of music *chef's kiss
Wow! That was trippy and cool - amazing what can be done with today's software...and even more amazing, YOUR brain for coming up with these ideas and then managing to execute them too! This channel is a deep well of happiness for me and I'm so grateful you keep making things to share with us! Thank you!
won't be long before the AI starts coming up with these ideas as well
This is so satisfying. The video. The music. These images. This is what AI should be used for.
Kinda depressing
@@davidenorbedo5555 no u
Visualizer part was definitely the best, but generally the whole video just woah'd me!
Okay, I heard the expression "rolling hills" before, but can we also use it for mountains and/or islands?
Great video, as always! Very atmospheric. A little harsh on the eyes, at times, but a real treat nonetheless =)
As someone who's already seen and used this AI, I know what it's capable of, but was not expecting something this good. Great job, can't imagine how long it took to make this masterpiece!
That's absolutely beautiful. Your artistry never ceases to amaze.
Please do more of this, it pleased my synesthete brain SO MUCH!
Okay, that was really cool. I always enjoy your video's of you making some kind of way to sync with music, and this, out of everything you have made so far, has blown my mind. Keep up the amazing work DoodleChaos.
DoodleChaos getting into AI art is one of the things I didn't even know was on my wishlist for the world. Absolutely fabulous!
Usually, I avoid supporting people on Patreon when they're already a huge success... better to help those where my contribution makes things happen that wouldn't otherwise, you know?
But if you're going to get into hefty machine learning models, maybe you'll need it, haha.
How!, You make the most interesting and awe-inspiring videos EVER. This was different and in a very postive way. I love how the terraforming of and and sea synchronized with the electronic music to form eye-catching videos. Keep up the synchronization between upbeat music and the interesting visuals. :)
Its always a pleasure and an honour to be able to appreciate the videos you create for us to watch.
That said, I would love to see some of this new tech accompanied by some classical music like in the older videos.
Still, great videos :) thank you the hard work you put into them! :D
the reasons ive subscribed to ur channel is all the different and creative ways u show the music. This video was a great example. Cant wait for ur next video!
1:58 "KoOraih" wise words
Just like this ratio!
Love the lil river dabs 1:36
Fun fact: This isn't actually AI generated. The music is DoodleChaos's gigachad vocals and instrumentals, and he simply transformed mountains, rivers, and forests to get the images.
This is amazing! You're one of my favorite RUclips channels, I never know what to expect next but it's always so cool.
Thanks for amazing work! I especially love the part where your logo pops up :) (oh and floral equalizer, too!)
So cool!
My favourite bit was with the overview of the valleys. And when it "dipped" diagonally to the side 😄. Loved it
Another incredibly unique, extremely satisfying video to watch and listen too! Loved how you incorporated your logo 👍
good but nothing will be as beautiful and satifying as your works of absolute are you are the best at this doodle! i also didnt realise the stone henge thing whas the doodle chaos logo!.
It looks really good, congrats ! It's one of your best vidéo
And as always, doodle shoved us another very interesting approach to music!
Very nice job man!!
*Absolutely astonishing 😍😍😍*
The best part is that I could actually see a band creating a music video like this
At first, i didnt notice wut it was at 2:48 but then i realized for it was your logo picture. That... was pretty awesome if i do say so myself! Props to you man! :P
Damn, patrick's rock has grown much older in this video.
Wooow! What a creative idea! And it turned out amazing! Watching you play with different media is so cool and inspiring!
You keep getting better and better, love the change from what you normally do :D
I can't tell you how mesmerizing that was to watch. The beat of the music to the image's changes from frame to frame is strangely soothing while incredibly interesting.
I can honestly watch this a thousand times in a row and not get bored of it!
I would love to see another video like this from you someday, if you want to do it again.
12/10 Brilliant job!
Wow, this is one of the coolest concepts I've seen in a while! Very mesmerizing to watch!
Bro it's as if the nature is dancing wtf
Like the river part
Amazing!!!
And then those sound waves! Damn dude!
THIS IS AWESOME!! I wanted to make something like this for so long, to see it out in the world by a youtuber I love is amazing!
Absolutely fantastic work and great choices
Whaaat this is so interesting (like I've never seen something like this)
Incredible job! I think it puts my old experiments to shame!
Thanks for continuing to make such cool stuff!
This is super cool. It’s incredible how amazing your video are.
Great job! :) When you publish it always makes my day! You make the best music content in my opinion! Thank you for all you do for us! :) I appreciate all the time you spend on these video's and I'm sure a ton more people also do! :) Keep up the great work!
Thanks Enderglow, I appreciate that, have a great day!
@@DoodleChaos Thank you so much!!! :)
This is so original and unique! What a cool idea. I can't stop watching.
I've been watching youtube since 2012.
I have seen a lot of things.
I've learned a lot of things about how different programs and skills work.
Generally even when I don't know entirely or are out right wrong about how something works I can still put forth a guess that I'm happy with.
This is the first video I have seen in my entire life where I have absolutely no idea how it happened or how it worked.
Like... you can paint a pretty realistic cloud to go over water if your patient enough...
But then the grass and water reflecting trees... and the Trees!?
Those trees are so detailed it must have been computer. But where how does the stuff move between it so well... Agh, at the end you revealed a bit with the colors and shapes and I guess the computer fills in those shapes with landscapes... but how are... some of those landscapes have to be from actual nature outside... but there's no way enough real world stuff exists to let the animation be smooth...
I have no idea. This is amazing and I don't have to know how it happened. All hats off to you sir... or ma'am... or whatever you prefer
Very well done
I thought you did each fame by hand but the animation in after effects to an image sequence is a complete GENIUS idea. I don't know how you do it. This is incredible.
wow this is really taking up a notch! hope to see more like this
I needed that smile today... thanks for the hard work as always, Doodle
💐🐱💐
No matter what the medium, DoodleChaos shares his opinion to people through music and visuals without fail.
Wow! This must have taken a while!! And it looks so beautiful!
Damn this is so cool! I love everything about this! 💕
Incredible! :D
Good job adding your logo in there LOL
Very awesome video. 3:00 it took me a while to notice it was the doodlechaos symbol, but when I did I thought it was cool. Looking froward to more music syncs, and do like the use of NCS songs. I like a lot of NCS songs, but some I don't, so far you have used some I like!
(edit, ish. The moving rock I thought was odd, but it works. The clouds do work better for moving though.)
Great job man, this is mind blowing
Amazing stuff!😁👍
Your creativity is simply inspiring! Well done!
Yooo, the visualizer looks so cool!
I loved the plant EQ. The AI came up with some interesting interpretations for your animations. I love it.
Underrated and amazing. Great job.
3:20 the plants representing the frequency spectrum absolutely blew my fucking mind that was insanely cool
That! Was art! You really amaze me and I am in love with your work! Keep blowing peoples mind! You rock!
Now this is epic (and creative too, love it)
The AI does such a good job at making drawings into beautiful realistic landscapes 😻😻😻
Gorgeous and really cool!
pretty different, why not?
turned out better than i expected lmao
good stuff as always
Just amazing! Make more soon plz!
Never thought about using gaugan this way. Wonderful ! :D
I CRY that this only has 1/8 million views. DoodleChaos you are someone specially gifted. I don't know if ANYONE has done anything like this before.
Wow! This is incredible and underrated!
This is so cool. Subbed!
Really cool, I liked the part with the doodlechaos logo changing around!
Hey Jake!!!
That was really unique and cool, never seen something like that! :D
This is super creative and I love it!
This is awesome, never seen anything like it!
I was thinking about this same music today and see this... is just magic. Good work♡
The way it does this is so trippy! Corridor Crew needs to see this
Holy shit what a cool idea, I could easily see something like this as a video playing for a live show. Gets my mind racing with all the possibilities
Love this song, Good work!
Interesting video! That was pretty cool!
This is amazing!!
This idea of use from the release of that ai is the most original i saw
Very nice job
It’s very amazing , you are so smart and ingenious ,this video is fire 🔥🔥🔥
more of this please xD its cool!
Wow what a video and what a song choice!
Damn, the audio spectrum part is so fascinating :O
every time I think you can't up your last video, you pull out something like this. Absolutely insane
This is AMAZING!
this is awesome, im def gna try this out for myself. u just put me onto something big, thanks bro 🤘🏻