Runway AI Gen3 : Video to Video Test + AI Thoughts
HTML-код
- Опубликовано: 2 окт 2024
- Hey all! Been awhile. This video is a couple things in one. An update on my status in all things film + Testing out the new Gen 3 Video to Video that Runway AI released and finally my thoughts in general on AI.
_______________________________
Subscribe to keep up to date with future content! Don't forget to hit the bell!
Follow me on other social media below:
X →→→→ x.com/robertdo...
Facebook →→→ / sfezfilms
Instagram →→→ / cellreborn
#Runwaygen3 #Western #AIVideo
I never imagined this would even be possible. I was diving into doing 3d shorts with Unreal but Ai really does get your imagination out faster. Awesome work Rob!
You know I saw some tests other people did using low poly, untextured animations made in blender ran through video to video AI.
I've been trying to learn Blender and UE myself so it's definitely something I wanna test out with how they can work together.
@@RobertDodrillUsing 3D for the base pass is definitely going to be helpful for scenarios where you still want to heavily art direct the camera movements or action scenes in Previs first. Then the Previs output can be fed through the video to video pipeline for a more presentable version with backgrounds and lighting and textures which may help the director and DOP or the art director make a decision much earlier in the process (and quicker). I think there will be some short term impact on jobs but hopefully in the long run, it’ll evolve into another toolset for artists.
Also, I just discovered your channel through this video. Subscribing as I like your take on this and opinion piece in the video besides just showing the workflow or the result. Cheers!
Incredible, its a game changer. We all going to be film makers at least ppl with imagination because not everyone has that lol don't forget that with other AI you can even create characters not and everything you can imagine. Like with Kling. I sai in about 3 to 5 years the technology will be so great that we will be able to make a movie as good as a Hollywood movie and no one will see the difference. And we will make that from our house alone.
Done like your video
All that missing is Loras for character consistency
You got a great voice, easy to listen to.
Upvote. Very interesting material. Thank you.
Brooo nice video please make moreeee
@@mrzzz492 keep an eye out tomorrow evening.
J/K it's up now because I forgot to hit schedule
i try to keep this short .. ai to me feels like just another nail in the coffin. there is not a single hollywood blockbuster what ever you wanna call it .. not a single movie within the last lets call it 15 years that was even remotly what i would call a "good" movie. so like story bad actors bad special effects .. here we go .. the more special effects became available to those making movies the more they just put in alot of special effects thinking that would make up a good movie up to the point where the movie itseolf just feels like advertisment of a new gfx card or soemthing. witht that beeing said now ai comes along as if the remaining so called film maker arnt bad enough as it is already now ai comes along and helps making the output even worse. it might sound exagurated but i could just vomit if i see ai created content its like talking to a answering machine. they are not designed to actualy relay the information they dont even get checked they are nothing but loops to keep customer entertained or in a state of .. oh yeah i reached support .. meanwhile no one will ever listen to what they talked into the phone on that answering machine. its just a waste of time like watching ai content or listening to ai content. is it just me that i think it is so bad ? am i just getting cranky or old or both .. i have no idea where this is going but as i stopped using phones like more then 20 years ago i also stopped watching movies .. first it was like trying sequals some came out ok lets take terminator 2 for example or rambo 2 .. but what came after and im not even talking godzilla 38 (i belive we are at 38 not that sure) or just another king kong rocky 15 rambo 20 starwars 30 i donno how many did they sneak in there already .. and one just worse then the one befor. do i just have antequated taste or something i dont know :( sorry spamming your comment box but as you said you having your channel not in sight of algos my comments are propably not beeing read by the average public anyway so kinda fits :D and since im from germany forgive me my mistakes i hope its not too bad but as long as its not as bad as ai .. well at least they acomplished the goal like how to make even worse content and work even less for it as well. and the youth so to speak they get used to the absolut low level of quality. how can they even tell if its bad if they dont even know other movies like .. we could pick so many examples lets take just for fun the audio title you used in your clip .. ennio morricone i belive it was this was a "machine" .. capable skilled simply said he was good his work was worth something ... i miss that today i miss that for quite some time now and now ai comes along and as i said at the beginning is jst like another nail in the coffin.
thats my 2 cents on that topic with that beeing said everyone who actualy read that .. dont let that stop you from making movies .. only if there are people who are bold enough to talk themselves into beliving the farytail we all like to tell ourselves you know the one that goes "i can do better" only if those people try .. even tho 999 will fail there will be again one who will actualy be worth something. maybe one day befor i die i get to see another movie that is actualy good and if someone has a suggestion like a movie that might be worth watching feel free to let me know .. i wont give up searching and i hope movie makes wont give up trying ... one thing forsure they cant get any worse as the movies are today
cheers :)
The issue is only Hollywood is making movies now truly creative people will get a chance.
@@Dayta I totally get what you're saying and I think I mentioned it where I said movies made with a real team of humans will always be better than AI driven ones.
Still requires a good cinematography. I consider it a spectacular relighting plugin.
Inspiring Stuff. Definitely gives me ideas...🤠
Yeah this really does change everything
That was an enjoyable test to watch. I appreciated how you could retain your likeness while changing the backdrops, I was afraid the AI would make too many alterations to your physical appearance so as to make you unrecognizable. Obviously I noticed a few very strange visual bugs here and there. I'm watching this stuff change literally every week so I imagine a lot of this will get worked out (remember back to Will Smith eating spaghetti?) I'm very interested to see where all of this goes.
I would literally sit and watch this as a feature from end to end the directing, lighting,acting, and over all cinematography are very believable and engaging.
I am impressed !
I am so excited !
I can't breathe !!
😂👍
As a short filmmaker myself, I appreciate this. Keep it up man.
You are not short, you are average sized.
for what it is, this is mind boggling, great job alone on the cinematics
Me: Not sure about AI 🤔 Watch This: 🤯
in few yrs u r weraing googles, and see the world as you want it
This is f*^%$ amazing!
i enjoy it good work mate
Nice job brother
Prompts please 🙏
Hi, Robert. Very nice results, congrats! I have a very extense project with 8 books, "Blue Angel Series". I could only make 3 animated episodes so far, from a total of 60+. AI will be extremely convenient to make it using 3D as a base. I'm particularly interested in lipsync. Did you use Runway's lipsync tools or Gen3 generated the lips movement with video-to-video? Thank you!
For me in the original clip I actually said "I'll find the man who killed your father." But then when it generated the mouth didn't really sync with father, but it did seem like it was mouthing close enough to "pa" which worked for the western setting. So I just did ADR and re-recorded the line to say "I'll find the man who killed your pa"
So, I suppose the lipsync to the original video was about 90% the way there just by the initial video to video conversion.
Lip sync though might be a good idea for additional tests, so I'll think about an interesting way to approach that.
Astonishing results. Looks like you had fun, cowboy! Cheers.
are you aware that at 3:32 you finally describe what you PLAN To talk about. instead of just starting the video and talking about it, you blame the algorithm for your inability to retain viewers. self awareness,
@@breakfasttouch No, I wasn't aware that I planned anything for this video at all.
Yeah he lacks awareness of self. He's apart of the problem with AI and technology because he wants to take away jobs of hard working Americans. People like him are the reason why Ford and Toyota are making record profits.
We need to stick together to keep cinema to the blue collar workers and stop this nepotism.
Your testing really shows what the current possibility’s and limitations are. I agree with you that as of now it’s not quite up to hi end production movie quality. But! It’s definitely on it way. If you’re at the level of having resources to hire people to make this better then by all means. For a pitch trailer this would be spectacular! I pitch tv shows and having this to show “proof of concept” is fantastic.
I'm an FX artist (part-time) as well, and what I've observed with this (personally), is that it doesn't necessarily have to take a person's job/livelihood away from them, but they could use this tech for scenes (sparingly) that would be either difficult or extremely time-consuming to accomplish.
I don't fully "embrace" it (i really don't like that word sometimes), but again, this is a personal reflection on it. I think that it could take away jobs (sure), but again, if the industry doesn't go full-on with it, and allow the animators/FX artists to use it within their workload, then it could be beneficial (to an extent).
Side-thought: on a (kind of) different note, what if game developers/console companies (ie. Sony, etc.) would use this tech (once it's sustainable) for games in the future?
@@Entolyfik Being a bit of a Trekkie, I honestly hope to see a day in my lifetime where you can put on a pair of VR goggles and enter what amounts to a Holodeck, and a world is generated of whatever you ask.
@@RobertDodrill OH man, a Holodeck?? That would be incredible!
When I was younger I wanted to invent something of that nature (based on Batman: Beyond...similar premise).
I would totally like to see that...Just imagine: you put the headset on, speak what you would like to see, and there it is, AI generated ("endless" procedurally generated world) upscaled in 8K (16K one day??), People don't have 8 fingers (unless you want that, lol), and the realism is so prevalent that you can see even a speck of dust rolling down inside of a bead of sweat on someone's face!
@@RobertDodrill Picard would be proud, lol!
@@Entolyfik It would be amazing! I feel like all the pieces of the puzzle are there now and possible they just need to progress. We have video, we have language models people have incorporated into NPCs, and recently things like Google Genie and I forget which video game company showed off playable AI. It feels like it's only a matter of time at this point.
How do you make the result image look like you? it gives me someone completely different
@@SerendipityForever I'm thinking it has to do with how close the shot is. Because the wider shots lose detail in my face.
My next test is going to be a dialogue scene with close ups, and then some wide shots to test if I just got lucky or if there are other settings involved. Also to test lip sync.
@@RobertDodrill I saw another comment about that so I tried a close up shot and it still gave me someone completely different. For the record I'm Asian and it gave me a white guy lol perhaps it doesn't have enough data on people that look like me... would love to see a step-by-step video of you doing it
@@SerendipityForever I know that with Midjourney for image generation that it's helpful to include the race of the character you're trying to generate, if you want a specific race. So perhaps when you're describing yourself/your character for the prompt try fitting in Asian male as a description. That might push it to scan or whatever it's doing more accurately. Just a thought.
@@RobertDodrill That's a good idea, I'll give that a try! All the best to your channel
AI will not only do as good as current hollywood 200 million dollar productions, it will surpass it with the ability to do things you could never dream of. Because you will be able to try or do things you wouldn't try today because of cost. The quality is not there now, but I'm doing concepts that would be a waste of money to do, completely ludicrous ideas that are way out there. It's so much fun to be able to do things you thought about but never could do, now I can try 10 different ways.
People are not seeing what's coming, I remember artists a little over a year ago saying no way you will have the quality or style of real artist, and no way it will it ever be creative. They were 100% wrong, AI art is better in every way, it surpassed real artist and yet it's still evolving. AI video next year will be jaw dropping, it will make current AI video look like garbage in comparison.
For real they were coping so hard… AI got them scared
I have tested it. It was not that good really. It kind of sucked actually.
But the results here are great. So what gives? Potentially image sharpness, resolution, and shutter speed may all be contributing to loss of detail in video to video conversion.
I remember last year when @mreflow (Matt Wolfe) had only a 5k subs... He kepth at it and now he is getting close to 1mil. Start doing tutorials and talk about AI movies. You'll reach the 1mil subs by next year.
@@JavierCamacho That's pretty cool to know. Definitely has been getting a good reaction and got me thinking of more tests. I wanna try green screen test with it but also I'm probably going to write a short dialogue scene and see how I can get lip syncing working. Trying to think of ways it can be used more practically with no budget filmmaking.
5:24 This one was my favorite, good acting dude, and the lighting in the background. Very interested to see how runway will be in a year.
@@RealBefuddling same!
You're in the algorithm now dude
@@GriefBurrito Now it's only a matter of time before League of Legends reaches out to me, then I know I've made it.