I have an idea to make it better. When you create character's 3d model Focus on detail of characters head and face . Put it into unreal engine 5.2 and Resolve Metahumen identity. After that Use your iphone to best facial capture. And use mixamo to body animation. I think you will love that workflow .
It's really refreshing seeing someone diving deep with all the current tools and go through workflows on a good speed. No Mumpitz here, your channel is a little gem, very inspiring! Runway ML is really milking it hard, thanks for not only sharing all the commercial solutions like most others.
I tried that, but it didn't work very well because of the low resolution. EbSynth often has trouble making Lipsynch look convincing anyway, and the results I got looked better and were less work. But I will take another look at ebsynth (especially Tokyojab's workflow) for another project!
That’s amazing! It’s interesting that we learn the tools, but there’s always something new on the way you use them that’s completely different and does more than we expected. I am definitely becoming a patron today! Thanks a lot for sharing!
Man, your videos are truly amazing. I'm just dipping my toe into AI video, animation, and editing, as I'm working on a proof of concept for an 80s-style creature feature. I will definitely track your successes as an inspiration/method for this project. Thanks 🙏
I love this video - what would improve it is a list of softwares you used, in order, with a price for monthly subscription for each. As a hobbyist trying to forecast if this process is a worthy investment, the numbers would really help. Thanks again!
I am amazed by how consistent you got Deforum. If it wasn't for some of the left over flickering, I would prefer it over Gen 1. If you combined it with Tokyojab's workflow for Ebsynth with Stable Diffusion, I think you would have knocked it right out of the park (for free, but a lot of work though). Thank you for all the work you're putting in and how well you document everything. Always look forward to your videos. Amazing quality! Keep it up bro!
Yeah wow. I think this is the future too, and it's awesome to see your prototyping at this super early stage of the game. I imagine "in the future" there will be programs dedicated entirely to this process that are as ubiquitous as photoshop. It is your tinkering now, that paves the way to that future. I am going to sub to your patreon to get you a new graphics card! Poor thing must be melting! Thanks for doing what you do.. I am on the same journey and have basically been devising the same workflows as you, but thanks to your work, I don't have to test them all out. Its awesome and I can't wait to see the film. Much Love from the land down under :)
i use the Quad Remesher addon for blender (like $89) to get better topology on the character mesh, then I use SoftWrap with a custom face/head mesh, to redo the head. then we can use another addon I haven't gotten yet that generates blendshapes for the character in blender.
Great stuff! Your workflow videos are clear and also relaxing. Thanks for exploring these processes and sharing withe the void. Definitely agree that animation and even filmmaking will eventually include an "AI pass" at the end. Similar to color correction and sound mastering. So cool.
Thanks for sharing. Love your bandwidth. And I also agree, that its always best to consider supporting the open and free software, like a1111 or deforum as we can all grow from it! rock on!!!
Thanks for sharing your workflow its is so creative and really useful. The final result May not look pixar, but it is on the way, lets say to a 65% which is unbelievable Because Just one person can do it in record time. This is the future man..
Thanks ❤ your other videos a few weeks before where very much Inspiration for me. Ive tried and learned extrem fast. Now i am almost on the point that i can compose my whole pieces from all these amazing tools i used (yes also payed did and wonderdynamics, runway....) i am a hobby creator and a single dad of a 7 year girl. 6 Month before i only got a smartphone and cuttet some Rapmusicvideos of my own Hobby(parent)-gang. And now i am on a point that i turned my created avatar into inreal engine. Looks great and moves 😊. I see how fast this development improves workflow and getring Ideas come to reality. Ive not uploaded so many stuff but i would be honored id you check these few vids. As i watched your videos i haven't even heard the word depthmap, but was so impressed by your ideas, that i am annoying anyone in my life with things they don't and pretty sure won't hear about it. My big Project is called "The point of convergence" A fantasy storie which is about a fictional me and the world of my friends. And the (try and error but learning)-project is an Androidstory(oh so creative😂 i know😅). Besides i step into learning comicanimation with cute characters for my daughter. I produced a learning video used free version of adobe character animater, canva pro and filmora 12 pro. Think will upload it these days on my second channel because it stands out pretty much from my Homechannel. For my android Story i used did to sync, but spoke it myself(and friends) with VOICEMOD pro and recorded and finetuned it with Reaper and IzotopeNectar (which is the only programm and vstplugin i was familiar before i stepped in this crazy mindblowing ai world😅) feel free to visit my channel. You can also see my beloved little daughter in her first role playing in front of an Greenscreen almost 2 Years ago. (No panic at all Webhoodwatchers: playing a role, coverd in her acting as this person and facecoverd because i am not a ****. Looking forward to see your next stuff. Because if i see these voluminous golden hair and hear that smooth English voice, that i was teached in german school, i know i will be illuminated with knowladge for weeks. Not sure why i dont take a translator or an ai😂 so excuse my mistakes in writing (on smartphone) or grammer. The "idocrazy" way of life is possible near, so i think use our brains so long as we are allready know that we want to use it by ourselfes. Good Night good Life. Have to rest its 03:27 am😢
This is amazing! You just made me realize that I can save thousands of dollars for my project, I never realized that it was possible to do this on my own, thank youuuuuu 🥰😀
@@ManuelJimenez1 I didn't mean it like I already had the money for my project, I meant it as like "I don't have to wait to have a way of funding these details in my project to start it". I'm a student lol, I only have enough to survive, but I was already planning on subscribing to his Patreon page, because he gave me what I needed to be able to continue my project, because I was seriously about to drop it. Anyway, it's cool to see that you already subscribed to his Patreon, that helps support him 😊
when the AI processors are powerful enough the graphics cards will have two stages the 1st will generate a low poly image and then the 2nd stage (generative AI) will generate from this image a high resolution image with the style of an artist and in real time
Respect for exploring what the tools can do but there seems like so many steps I wonder whether a professional animator / CG artist could achieve what you're doing at the same standard (if not higher) in the same amount of time?
That is what i am planing to do..... sad but happy someone done it before me I also suggest to use 2k2k rather than ICON cause 2K2K is better on skirt. And, it seems cloak and hat is such a chanllege in 3d modeling...
extra-ordinary ! mickmumpitz what a research ! as story development pro I can see the possibilities but this video results shown - at this point -suggest that it would be a lovely 'bad transmission' video to actors within a narrative. perhaps cutaway scenes in videogame - for instance: a 'lost video diary'... 'its impossible' has a too far extreme head movement that an animator would not do. the old man movements got better... but I guess tom cruise is not at risk today. aiming pixar at first shot could possibly be a bridge too far? but I believe! animating a full feature animated w your take on AI is just a bit further away - very possible [ but screenplay upgrade is a must ]. in my opinion trying to do ALL animation with AI now is going to make a result but not a longing delicious animation. if you tried to make a longer teaser scene using more camera work it might be less strenous to watch because the visual cinematic narrative would compensate for the visual weirdness noise. unfortunately a book from XIX requires extra screenplay adaptation and animation skills. on the other hand, check animation from Nat Film Board Canada has a very high level of arts experimentation, and some as Coe Hoedeman took opportunity from low developed characters to tell a very sophisticated story. why not try a funny monster that only has a noisy line to say ? another aspect is that animation is an art of transformations, so a person that shifts is likely to succeed as storyline. meanwhile, cheers from Rio. you said would like to see what we do: Bandolero ! short 5 min live action cartoon ruclips.net/video/OgckykwLjMg/видео.html I've used a cartoon 3d background +chromavideo into unreal.
It's interesting you're using a 2070 graphics card. I recently bought a second hand 3090 desktop PC to start my AI journey. It's significantly better than running on my M1 MacBook but I'm still yet to test it on video generation or anything beyond image generation. I've been struggling to find work and am trying to create a TV show using AI to help bring in some income until I find something more stable, or until it creates enough income to just keep doing it.
Could you please tell if you used the original output from blender directly to deforum and it gave you a good lip sync, without need to use all previous instruments to fix lip sync?
Lovely video, thanks for sharing! I've been meaning to test reprojecting the high resolution texture back onto the lip sync after wav2lip with Ebsynth but haven't got there yet. Would love to know if anyone's tried this and/or encourage others to. I also have a concept for a node setup in DaVinci Resolve that will track a face and export a cropped image sequence for external editing (for example, working with Stable Diffusion or Ebsynth with smaller files, but maintaining higher local resolution)...
No idea what's happening. I would've used Vegas for the lip-sync battle... so prolly outta my little pond of expertise. That said... FABULOUS! Something there for everyone! See? Why can't banks be like you... just giving away cash-knowledge...
U make everything look so easy to do... thanks, been looking for exactly this, how do u make ur own animation using AI. might need AI to take over my brain to do this anyhow, but I'll give it a try.
hello friend, I have a few unpublished and unfinished graphic novels, and I would like to learn how to create animations with AI to bring my graphic novels to life. I would like to know if you had consider to do private classes, how much you would charge. Regards, Clau.
Hey, that was absolutely amazing. Thank you so much for providing such value. Do you also teach over Zoom and offer one-hour classes? I would definitely be willing to pay for this. Could you provide your contact information so that we can discuss it further?
It’s still quiet far from everything. There are already better non ai tools that can do all this extremely well. It would take people to work specifically into developing software just for the purpose to mix all this together and make it accessible. In other words, people with a large pocket that can afford to use money in high risk high reward investments. So for the next three years, it’s going to be an all out war until all this ai software can communicate well with stablished or new edge applications. The sooner the better.
Before I sub? Can you share how many hours in TOTAL the entire workflow took to finish this 15 minutes Vid? Oh, what the heck!!! Sub'd and Liked 👍 For all the work YOU'VE 💪DONE!
Need advice how I can make a character still if I'm in a wheel-chair. I would definitely struggle to get into that T-Positition... Would love some advice please? … Kyle x
Thx a lot ! I’m working on an ai series on RUclips and your video will save me a lot of time! Thx again ! Great job!
I have an idea to make it better. When you create character's 3d model Focus on detail of characters head and face . Put it into unreal engine 5.2 and Resolve Metahumen identity. After that Use your iphone to best facial capture. And use mixamo to body animation. I think you will love that workflow .
It's really refreshing seeing someone diving deep with all the current tools and go through workflows on a good speed. No Mumpitz here, your channel is a little gem, very inspiring! Runway ML is really milking it hard, thanks for not only sharing all the commercial solutions like most others.
nice to see you keep watching new workflows all along mix em and then share it with us
Incredible display; what an exciting time for independent creators!
You can use a still shot of the character's face with EbSynth and make it so the style doesn't change at all while they're talking.
I tried that, but it didn't work very well because of the low resolution. EbSynth often has trouble making Lipsynch look convincing anyway, and the results I got looked better and were less work. But I will take another look at ebsynth (especially Tokyojab's workflow) for another project!
@@mickmumpitz Good to know. Did you try just using the first frame of the character, which was the original image?
This is insane! AND it will only continue to improve
pretty cool.. keep up the great work
The lip sync idea for the texture was sooooo fucking smart!!!!
That’s amazing!
It’s interesting that we learn the tools, but there’s always something new on the way you use them that’s completely different and does more than we expected. I am definitely becoming a patron today!
Thanks a lot for sharing!
thanks friend. i always wait for your videos
Oh, it’s so amazing😮 wonderful job
Man, your videos are truly amazing. I'm just dipping my toe into AI video, animation, and editing, as I'm working on a proof of concept for an 80s-style creature feature. I will definitely track your successes as an inspiration/method for this project. Thanks 🙏
Amazing work! Congratulations man
Holy moly, that intro trailer is just incredible! Fantastic work.
I love this video - what would improve it is a list of softwares you used, in order, with a price for monthly subscription for each. As a hobbyist trying to forecast if this process is a worthy investment, the numbers would really help. Thanks again!
Great job man! Thanks for all the hard work of creating these videos.😊
Someday there will be a single program that will do all that
I am amazed by how consistent you got Deforum. If it wasn't for some of the left over flickering, I would prefer it over Gen 1. If you combined it with Tokyojab's workflow for Ebsynth with Stable Diffusion, I think you would have knocked it right out of the park (for free, but a lot of work though). Thank you for all the work you're putting in and how well you document everything. Always look forward to your videos. Amazing quality! Keep it up bro!
absolutely stunning video as always
amazing tutorial! thanks for sharing your expertise!
Yeah wow. I think this is the future too, and it's awesome to see your prototyping at this super early stage of the game. I imagine "in the future" there will be programs dedicated entirely to this process that are as ubiquitous as photoshop. It is your tinkering now, that paves the way to that future. I am going to sub to your patreon to get you a new graphics card! Poor thing must be melting! Thanks for doing what you do.. I am on the same journey and have basically been devising the same workflows as you, but thanks to your work, I don't have to test them all out. Its awesome and I can't wait to see the film. Much Love from the land down under :)
This is all happening so fast. Excellent videos!
Sehr cooler Workflow, vielen Dank für's teilen :)
You are crazy! This is awesome and what I've been looking for! Very impressed here and thankful
You are cray, man! great job
thanks so much for sharing. your description is easy to follow and your edits are entertaining to watch. great job.
i use the Quad Remesher addon for blender (like $89) to get better topology on the character mesh, then I use SoftWrap with a custom face/head mesh, to redo the head. then we can use another addon I haven't gotten yet that generates blendshapes for the character in blender.
Thank you for this tutorial. ❤ Very helpful.
Bester YT Kanal seit langem! 💪🏽🫶🏼🤝🏽 #1000thx for sharing und die Einblicke in den Worklflow… #ehrenmann
Brilliant work!! I totally love watching your videos, thank you for sharing and can't wait to see what else you have coming!!!💯🔥🔥🔥
Phenomenal video. Thank you.
Great stuff! Your workflow videos are clear and also relaxing. Thanks for exploring these processes and sharing withe the void.
Definitely agree that animation and even filmmaking will eventually include an "AI pass" at the end. Similar to color correction and sound mastering. So cool.
just saw the intro and thought: mega!
You make everything sound so easy LOL
Thanks for sharing. Love your bandwidth. And I also agree, that its always best to consider supporting the open and free software, like a1111 or deforum as we can all grow from it! rock on!!!
The neck movement crack me up 😂
Damn so dope dude!
great work mate , keep up
Thanks for this informative video👌🏾
you're the smarted guy on youtube😇
Thank you for sharing!
Amazing!!! Thanks a million bajillion!
tremendous work!
Did you check "runway GEN-1 alternative"?
Thanks for sharing your workflow its is so creative and really useful. The final result May not look pixar, but it is on the way, lets say to a 65% which is unbelievable Because Just one person can do it in record time. This is the future man..
You are so talented man, you can use so many software and I'm still struggling with fusion in davinci resolve
learning some new shit today👍
Thanks ❤ your other videos a few weeks before where very much Inspiration for me. Ive tried and learned extrem fast. Now i am almost on the point that i can compose my whole pieces from all these amazing tools i used (yes also payed did and wonderdynamics, runway....) i am a hobby creator and a single dad of a 7 year girl. 6 Month before i only got a smartphone and cuttet some Rapmusicvideos of my own Hobby(parent)-gang. And now i am on a point that i turned my created avatar into inreal engine. Looks great and moves 😊. I see how fast this development improves workflow and getring Ideas come to reality. Ive not uploaded so many stuff but i would be honored id you check these few vids. As i watched your videos i haven't even heard the word depthmap, but was so impressed by your ideas, that i am annoying anyone in my life with things they don't and pretty sure won't hear about it. My big Project is called "The point of convergence" A fantasy storie which is about a fictional me and the world of my friends. And the (try and error but learning)-project is an Androidstory(oh so creative😂 i know😅). Besides i step into learning comicanimation with cute characters for my daughter. I produced a learning video used free version of adobe character animater, canva pro and filmora 12 pro. Think will upload it these days on my second channel because it stands out pretty much from my Homechannel. For my android Story i used did to sync, but spoke it myself(and friends) with VOICEMOD pro and recorded and finetuned it with Reaper and IzotopeNectar (which is the only programm and vstplugin i was familiar before i stepped in this crazy mindblowing ai world😅) feel free to visit my channel. You can also see my beloved little daughter in her first role playing in front of an Greenscreen almost 2 Years ago. (No panic at all Webhoodwatchers: playing a role, coverd in her acting as this person and facecoverd because i am not a ****. Looking forward to see your next stuff. Because if i see these voluminous golden hair and hear that smooth English voice, that i was teached in german school, i know i will be illuminated with knowladge for weeks. Not sure why i dont take a translator or an ai😂 so excuse my mistakes in writing (on smartphone) or grammer. The "idocrazy" way of life is possible near, so i think use our brains so long as we are allready know that we want to use it by ourselfes. Good Night good Life. Have to rest its 03:27 am😢
This is amazing! You just made me realize that I can save thousands of dollars for my project, I never realized that it was possible to do this on my own, thank youuuuuu 🥰😀
They guy have a Patreon running, time to share if you saved a hundred dollars, right?
@@ManuelJimenez1 I didn't mean it like I already had the money for my project, I meant it as like "I don't have to wait to have a way of funding these details in my project to start it". I'm a student lol, I only have enough to survive, but I was already planning on subscribing to his Patreon page, because he gave me what I needed to be able to continue my project, because I was seriously about to drop it. Anyway, it's cool to see that you already subscribed to his Patreon, that helps support him 😊
@@ManuelJimenez1 Also, can I respectfully ask why is your comment so passive aggressive? I don't believe I said anything bad.
AMAZING！！！LOVE！！！
when the AI processors are powerful enough the graphics cards will have two stages the 1st will generate a low poly image and then the 2nd stage (generative AI) will generate from this image a high resolution image with the style of an artist and in real time
This is amazing do you think I can use the same software and techniques to create a music video?
Respect for exploring what the tools can do but there seems like so many steps I wonder whether a professional animator / CG artist could achieve what you're doing at the same standard (if not higher) in the same amount of time?
The best!😮
15:05 well... I think it's pretty close! And i also think that Pixar and companies alike are already exploring these possibilities. I mean, why not? 😃
That is what i am planing to do..... sad but happy someone done it before me
I also suggest to use 2k2k rather than ICON cause 2K2K is better on skirt.
And, it seems cloak and hat is such a chanllege in 3d modeling...
extra-ordinary ! mickmumpitz what a research ! as story development pro I can see the possibilities but this video results shown - at this point -suggest that it would be a lovely 'bad transmission' video to actors within a narrative. perhaps cutaway scenes in videogame - for instance: a 'lost video diary'... 'its impossible' has a too far extreme head movement that an animator would not do. the old man movements got better... but I guess tom cruise is not at risk today. aiming pixar at first shot could possibly be a bridge too far?
but I believe! animating a full feature animated w your take on AI is just a bit further away - very possible [ but screenplay upgrade is a must ].
in my opinion trying to do ALL animation with AI now is going to make a result but not a longing delicious animation.
if you tried to make a longer teaser scene using more camera work it might be less strenous to watch because the visual cinematic narrative would compensate for the visual weirdness noise. unfortunately a book from XIX requires extra screenplay adaptation and animation skills.
on the other hand, check animation from Nat Film Board Canada has a very high level of arts experimentation, and some as Coe Hoedeman took opportunity from low developed characters to tell a very sophisticated story. why not try a funny monster that only has a noisy line to say ?
another aspect is that animation is an art of transformations, so a person that shifts is likely to succeed as storyline.
meanwhile, cheers from Rio.
you said would like to see what we do: Bandolero ! short 5 min live action cartoon ruclips.net/video/OgckykwLjMg/видео.html I've used a cartoon 3d background +chromavideo into unreal.
Wow! Grossartig!
Sehr sehr sehr cool!!! Danke für die tolle Tipps, ich schulde dir, big time! Wenn du willst, dann möchte ich dich ein paar Runden auf Bier kaufen haha
It's interesting you're using a 2070 graphics card. I recently bought a second hand 3090 desktop PC to start my AI journey. It's significantly better than running on my M1 MacBook but I'm still yet to test it on video generation or anything beyond image generation. I've been struggling to find work and am trying to create a TV show using AI to help bring in some income until I find something more stable, or until it creates enough income to just keep doing it.
Could you please tell if you used the original output from blender directly to deforum and it gave you a good lip sync, without need to use all previous instruments to fix lip sync?
thanks a lot !
Thanks for ai❤
Lovely video, thanks for sharing! I've been meaning to test reprojecting the high resolution texture back onto the lip sync after wav2lip with Ebsynth but haven't got there yet. Would love to know if anyone's tried this and/or encourage others to.
I also have a concept for a node setup in DaVinci Resolve that will track a face and export a cropped image sequence for external editing (for example, working with Stable Diffusion or Ebsynth with smaller files, but maintaining higher local resolution)...
No idea what's happening. I would've used Vegas for the lip-sync battle... so prolly outta my little pond of expertise. That said... FABULOUS! Something there for everyone! See? Why can't banks be like you... just giving away cash-knowledge...
U make everything look so easy to do... thanks, been looking for exactly this, how do u make ur own animation using AI. might need AI to take over my brain to do this anyhow, but I'll give it a try.
How many did it cost for the total procedure ?
great! have you ever tried nvidia omniverse audio2face for lipsync?
Thank you! Yes, I actually did in my first video "How I generated a full CG Short Film with AI".
Does ICON work on a mac
i made 3d models in daz studio, we can use them in blender using their ears and some hairstyles skipping all those steps
genius!
What would you recommend for creating animal models? I need rats, spiders, Akbash dog, etc.
ooo i wonder how much gen-2 would improve this.
Hi How about do you use CSM to make back of character?
Csm is great, but the quality of the characters is unfortunately not comparable to Icon yet. But I already use it for many other 3D models!
cool techniques ¡¡
hello friend, I have a few unpublished and unfinished graphic novels, and I would like to learn how to create animations with AI to bring my graphic novels to life. I would like to know if you had consider to do private classes, how much you would charge. Regards, Clau.
Hey, that was absolutely amazing. Thank you so much for providing such value. Do you also teach over Zoom and offer one-hour classes? I would definitely be willing to pay for this. Could you provide your contact information so that we can discuss it further?
I am very interesting animation programming but I don't know idea animation program software,
wow!
4:49 free tool for ai face animation
It’s still quiet far from everything. There are already better non ai tools that can do all this extremely well. It would take people to work specifically into developing software just for the purpose to mix all this together and make it accessible. In other words, people with a large pocket that can afford to use money in high risk high reward investments.
So for the next three years, it’s going to be an all out war until all this ai software can communicate well with stablished or new edge applications. The sooner the better.
Before I sub?
Can you share how many hours in TOTAL the entire workflow took to finish this 15 minutes Vid?
Oh, what the heck!!!
Sub'd and Liked 👍
For all the work YOU'VE 💪DONE!
Hahaha you are the man.
Icon doesn't work anymore :( so can't even follow this tutorial haha... :(
This is what I was searching. Thnks
Need advice how I can make a character still if I'm in a wheel-chair. I would definitely struggle to get into that T-Positition... Would love some advice please? … Kyle x
Imagination is dead.
heheh Looks like Tom Riddle from harry potter. (his 16 year old version)
HOW do you know all of this?? AI has only been in business for 15 minutes!?
Your content is great, but the clickbait thumbnail is off putting. The content is strong enough to stand on its own.
not so good final result
Oh, hope your toothache are better now....
looks may deceive
why not just sculp by hand, in the end
my opinion, is horrible, and not efficient yet.
Yet... :)
Or you could just use stable diffusion...
Fix this fix that.. urgh..
TOO MUCH WORK. need an ai which can do that it one click. lool
Man….in the end, after all those hours of effort and money, I have to be honest and say: it looks like shit :/
Outstanding! thank you for your hard work!