Since people are facing forward, you could probably split screen and intercut the different jury members, so that more than four talk, by recording it twice (for now cropping out the first 4 so the next 4 are the “new” first 4). Then splice the shots together in the order you want, blending the portions of the shots together in the video editor or any program with compositing (perhaps something free like the basic Da Vinci Resolve:). Maybe when blending together you can take reaction shots from different takes at different moments, so everyone stays lively (feathering the edges between characters to blend the whole scene together and keep it lively/to sell it as real). Very good tutorial!:)
I use Runway and Hedra for lip sync. Runway team MUST take a look at Hedra because is awesome. Runway lip sync works better with an animated video of a person it is too static with a sigle pic (you must have a pay account), hedra uses a single pic of a person and gives you back all body and facial animation with lip sync and AI voices (512x512 resolution, free account).
@@aivideoschool Actually, I have an idea for you… Let me know if I should quickly post a zoom link for sometime today or tomorrow. I would delete it from the comments after that, but let me know if there would be a good time for a quick zoom with screen sharing:)…
This was a really well developed video and to use Luma to tie it all into that one scene was next level. Luma uses mid journey which is the best image generator in my opinion
Thank you! It's genuinely exciting to see how much has changed in less than two years. I feel like Flux to Luma/Kling/Gen-3 to Lip Sync is such a good workflow right now.
I just uploaded a short test of me morphing into Tony Soprano and then using LivePortrait to animate the face exactly how I wanted it to. And it works 10x better and looks more realistic. But the only issue is that you have to track it to your face in After Effects or another software
@@aivideoschool Thanks! I took a meeting today with an app working on over the shoulder lip syncs, etc, so hopefully things continue to improve at a rapid pace 🤞🏼😊
i didn't know about this feature . thanks for making this video . a question- can we upload our own voice in it like the voice cloning feature . can we do that?
You can upload your own voice like I did 00:08 but I believe you need to do the voice cloning in ElevenLabs. You are able to do voice-to-voice in Runway, so you could speak with your inflections etc and transfer it to sound like one of their voices.
Hi, I really appreciate your videos. I had a question about lip sync if you can help me. I until a year ago used to use an app (reface app) that had lipsync, you could talk over a video for a few seconds making the people in the video say what you wanted. Some time ago they removed lip sync and I am not finding a viable alternative. Can you tell me if runway gen 3 has the ability to insert its own video and give the characters in the video their own phrases to say? If runway doesn't do this can you tell me if there are any alternatives? thanks for everything.
If it cant handle a slightly angled face like at 14:15 then its not ready for filmmakers just yet as over the shoulder shot is the core of storytelling and this may not get that right however were headed in a positive direction for sure. Another year or two and wamn
Totally agree. A few months ago I tried an establishing shot with two people then cut to each of them in an over the shoulder style shot. Looking at it now, it's amazing how much better the lip sync is. But I think the over-the-shoulder shot might be one "cheat" for filmmaking with the current limitations. ruclips.net/video/nL9UowfT0PE/видео.htmlsi=p7T4JqO36_vFuYRh&t=557
@@aivideoschool I started following you some time ago with the same interest on making stories by leveraging AI, but it didn't take long for me to realize that technology just wasn't there yet. So I turned a completely different direction and decided to dive deep into learning 3D animation in the likes of iclone and unreal engine. My gamble was by the time I Master these tools, AI would have made enough advancements that I'll be able to combine the two to get perfectly Controllable characters, environments and camera angles while extracting the realism from generative AI. I have faith those two will meet soon and by then I will have everything ready on my side 🚀
Heyy brother love from India ❤❤ can you please make a video on how to make a alien like your thumbnail of this video by any free version of ai or please I want to make exact alien like yours please make a tutorial on it
Hello, I'm glad this video has a fan in India! I used Flux for the image generation. Here is the exact prompt I used for that image: "cinematic medium close up of a bearded mechanic and a female extraterrestrial humanoid alien wearing a demin jacket walking down a road side by side, in the background is the bar they just left, their faces are lit by a street lamp they're passing under" This is probably the part you're looking for to get similar characters: "a female extraterrestrial humanoid alien" Here is more information about Flux, which is free if you have the right hardware: github.com/black-forest-labs/flux
As good as it is, it still can't do animal faces talking. If it does that then it will be something! Otherwise Pika and hedra are also really good at human face lip sync. I can't think of any scenario where more than two people will be talking n a singke camera shot without a change in camera angle. It's definitely a good display of capability, bit it does not have much practical application in film making. Even if you are able to show multiple people talking in a single frame without switching camera, it will look very unreal and mechanical. Only one scenario looks legit that is two people walking and talking facing camera in the same frame. Other than that there's not much of practical application. What is really needed is talking while turning head and animals talking.
100% to all of this. If you could control camera movement and have characters talking in profile, I could see how that might work for a scene with multiple speakers but we're probably a year or two away for that. (Watch it come out in three weeks now that I said that)
Hi I love your video but listening to your narration is a bit distracting since your voice seems to be unbalanced coming out more on the right side of the headphone... So other than that pls keep up the great work 👍
Thanks for pointing this out, I didn't notice when I was editing on my laptop speakers but I just looked in CapCut and the left channel is a little lower than right. This was the first audio I recorded with a Vocaster I recently got so it's probably a setting or dial I need to adjust there. Thanks for the heads up, I'll keep an eye out on the next one!
Runways lip sync is absolute garbage, they really need to invest more in improving it because gen 3 is so damn good, it's a shame the lip syncing aspect is very lacking and def not ready for anything that looks passable. Still images look extremely robotic, let's not pretend that it looks good. Hedra is miles ahead of them. If Hedra can release lip sync on video, it would be awesome.
That animation was a winner l, been waiting for someone to nail that. Thanks
Thanks! I really like that style and it actually works with lip sync
Since people are facing forward, you could probably split screen and intercut the different jury members, so that more than four talk, by recording it twice (for now cropping out the first 4 so the next 4 are the “new” first 4).
Then splice the shots together in the order you want, blending the portions of the shots together in the video editor or any program with compositing (perhaps something free like the basic Da Vinci Resolve:).
Maybe when blending together you can take reaction shots from different takes at different moments, so everyone stays lively (feathering the edges between characters to blend the whole scene together and keep it lively/to sell it as real).
Very good tutorial!:)
I think this split screen approach is a great tip!
I use Runway and Hedra for lip sync. Runway team MUST take a look at Hedra because is awesome. Runway lip sync works better with an animated video of a person it is too static with a sigle pic (you must have a pay account), hedra uses a single pic of a person and gives you back all body and facial animation with lip sync and AI voices (512x512 resolution, free account).
Totally agree with this!
Can Hedra export in different resolutions? I could only export square hmm...
Just got the unlimited! “Stay up til 3 generating videos”. You got me!!
Unlimited! I'm jealous. You are definitely not going to be getting any sleep any time soon.
@@aivideoschool Actually, I have an idea for you… Let me know if I should quickly post a zoom link for sometime today or tomorrow. I would delete it from the comments after that, but let me know if there would be a good time for a quick zoom with screen sharing:)…
There's an email button on my YT channel page, email that with info.
hi, do you know hơ to make the characters speak at the same time? im struggling
This was a really well developed video and to use Luma to tie it all into that one scene was next level. Luma uses mid journey which is the best image generator in my opinion
Thank you! I love when someone notices little details.
I love that Luma + Runway walking/talking scene! I totally get how that's an aha moment for you. Appreciate all your videos and enthusiasm 🙏
Thank you! It's genuinely exciting to see how much has changed in less than two years. I feel like Flux to Luma/Kling/Gen-3 to Lip Sync is such a good workflow right now.
I just uploaded a short test of me morphing into Tony Soprano and then using LivePortrait to animate the face exactly how I wanted it to. And it works 10x better and looks more realistic. But the only issue is that you have to track it to your face in After Effects or another software
Nice work, dropping the link to your test here: ruclips.net/user/shortsNYs4pOcg6AY?si=Upv-Mhsg7VGZGldA
Great epsiode. Always nice to see a tutorial on some of the lesser-talked about but massively important features on these apps 🤗
Thanks Tasha! Once they figure out how to get those profile shots talking, it's going to be a different game. Can't wait to see what you do next!
@@aivideoschool Thanks! I took a meeting today with an app working on over the shoulder lip syncs, etc, so hopefully things continue to improve at a rapid pace 🤞🏼😊
Nicely done!
Great channel you have!
Thank you, and welcome here!
oh wow, I didn't realize it could do multiple people, that's pretty cool
By the way, I love your channel, keep it up!
Very Awesome, thanks Brother!! Just subscribed!!
awesome, thank you for being here!
Great video!
Thank you, glad you enjoyed it!
Awesome
really enjoyed this, and learned a lot too. The link to your podcast doesn't seem to be working or is that just me? thanks
Thank you, I'm glad it was helpful! Also thanks on the link error, it should be working now.
how do you insert time/ pause
If you're using ElevenLabs voices, there are a couple of ways. Check out their prompt guide: elevenlabs.io/docs/speech-synthesis/prompting
@@aivideoschool thank you
Is there a way to lip sync to voice tracks you’ve recorded already? Thinking of using it to make an avatar for a speaker in a film. Any suggestions?
You can upload audio files in Runway and other lip sync tools to do exactly that. The lipsync doesn't have to be ai generated voices
@@aivideoschool thanks! Any avatar speaking apps out there? Soup to nuts replace a talking head with an avatar?
very nice thanks ^^
i didn't know about this feature . thanks for making this video . a question- can we upload our own voice in it like the voice cloning feature . can we do that?
You can upload your own voice like I did 00:08 but I believe you need to do the voice cloning in ElevenLabs. You are able to do voice-to-voice in Runway, so you could speak with your inflections etc and transfer it to sound like one of their voices.
It's a great tool to start with but they need to make it better cause it's a little glitchy.
Hi, I really appreciate your videos. I had a question about lip sync if you can help me.
I until a year ago used to use an app (reface app) that had lipsync, you could talk over a video for a few seconds making the people in the video say what you wanted. Some time ago they removed lip sync and I am not finding a viable alternative. Can you tell me if runway gen 3 has the ability to insert its own video and give the characters in the video their own phrases to say?
If runway doesn't do this can you tell me if there are any alternatives? thanks for everything.
Thank you! Runway will let you lipsync a video using an audio file or text (using ElevenLabs voices). I hope that answers your question!
If it cant handle a slightly angled face like at 14:15 then its not ready for filmmakers just yet as over the shoulder shot is the core of storytelling and this may not get that right however were headed in a positive direction for sure. Another year or two and wamn
Totally agree. A few months ago I tried an establishing shot with two people then cut to each of them in an over the shoulder style shot. Looking at it now, it's amazing how much better the lip sync is. But I think the over-the-shoulder shot might be one "cheat" for filmmaking with the current limitations. ruclips.net/video/nL9UowfT0PE/видео.htmlsi=p7T4JqO36_vFuYRh&t=557
@@aivideoschool I started following you some time ago with the same interest on making stories by leveraging AI, but it didn't take long for me to realize that technology just wasn't there yet. So I turned a completely different direction and decided to dive deep into learning 3D animation in the likes of iclone and unreal engine. My gamble was by the time I Master these tools, AI would have made enough advancements that I'll be able to combine the two to get perfectly Controllable characters, environments and camera angles while extracting the realism from generative AI. I have faith those two will meet soon and by then I will have everything ready on my side 🚀
I can totally relate to this 16:15
Inspired by real life
This can be done in other leguages?
ElevenLabs is multilingual so I imagine so. The voice selection might be more limited.
it looks so natural /s
Can anyone provide me with runway umlimited for lip syn?
You can buy unlimited credits and I believe that should give you unlimited lip sync?
@@aivideoschool can you help from where to buy it
It's free or paid
If it's paid how much subscription?
Paid, they have different plans.
hahahahah the end of your movie xDDDD
Heyy brother love from India ❤❤ can you please make a video on how to make a alien like your thumbnail of this video by any free version of ai or please I want to make exact alien like yours please make a tutorial on it
Hello, I'm glad this video has a fan in India! I used Flux for the image generation. Here is the exact prompt I used for that image: "cinematic medium close up of a bearded mechanic and a female extraterrestrial humanoid alien wearing a demin jacket walking down a road side by side, in the background is the bar they just left, their faces are lit by a street lamp they're passing under"
This is probably the part you're looking for to get similar characters: "a female extraterrestrial humanoid alien"
Here is more information about Flux, which is free if you have the right hardware: github.com/black-forest-labs/flux
Lip Sync is a cool tool, but it's a shame it only works with human faces.
I know, you'd think it wouldn't be too hard to adapt it to alien/humanoid type creatures at least.
@@aivideoschoolis there any other programs that can do non human faces?
His isn’t lip sync this is making characters speak text. Lip sync is requires an audio file.
As good as it is, it still can't do animal faces talking. If it does that then it will be something! Otherwise Pika and hedra are also really good at human face lip sync. I can't think of any scenario where more than two people will be talking n a singke camera shot without a change in camera angle. It's definitely a good display of capability, bit it does not have much practical application in film making.
Even if you are able to show multiple people talking in a single frame without switching camera, it will look very unreal and mechanical. Only one scenario looks legit that is two people walking and talking facing camera in the same frame. Other than that there's not much of practical application.
What is really needed is talking while turning head and animals talking.
100% to all of this. If you could control camera movement and have characters talking in profile, I could see how that might work for a scene with multiple speakers but we're probably a year or two away for that. (Watch it come out in three weeks now that I said that)
Hi I love your video but listening to your narration is a bit distracting since your voice seems to be unbalanced coming out more on the right side of the headphone... So other than that pls keep up the great work 👍
Thanks for pointing this out, I didn't notice when I was editing on my laptop speakers but I just looked in CapCut and the left channel is a little lower than right. This was the first audio I recorded with a Vocaster I recently got so it's probably a setting or dial I need to adjust there. Thanks for the heads up, I'll keep an eye out on the next one!
@@aivideoschoolGlad it was helpful to you. I know it's not easy to make a video but I'm thankful for your hard work 🙏
Runways lip sync is absolute garbage, they really need to invest more in improving it because gen 3 is so damn good, it's a shame the lip syncing aspect is very lacking and def not ready for anything that looks passable. Still images look extremely robotic, let's not pretend that it looks good. Hedra is miles ahead of them. If Hedra can release lip sync on video, it would be awesome.
you are too impressed with too little. This is too artificial to me.
It's okay to get excited about things that aren't yet perfect