Create Talking AI Avatar for FREE - Step-by-step guide (Spoiler: No D-ID studio)
HTML-код
- Опубликовано: 21 июн 2024
- In this video, learn how to create your own AI avatar with cutting-edge AI tools and techniques that are open-sourced and free to use. Our AI avatar was created using MidJourney, Chat GPT, ElevenLabels, and Wave2Lip, and you'll see how you can use these tools to bring your own avatar to life. We will not be using paid tools like D-ID studio for animating our Avatars. Get ready to explore the exciting world of AI and see what you can create with this AI Avatar Generator!
Links:
ChatGPT for script creation: chat.openai.com/chat
MidJourney for image generation: www.midjourney.com
Elevenlabs for audio generation: beta.elevenlabs.io
Wav2Lip for video generation: github.com/Rudrabha/Wav2Lip
Thin plate spline motion model: github.com/yoyo-nb/Thin-Plate...
Midjourney Prompt Used in the video:
"
a red headed woman with freckled hair and blue eyes, 30 years old, a character portrait by Ferdynand Ruszczyc, featured on cg society, naturalism, enchanting, pre-raphaelite, pretty.
"
☕ Buy me a Coffee: ko-fi.com/promptengineering
Timestamps:
Demo: [0:00]
MidJourney - image: [0:53]
2D to 3D: LeiaPix: [1:38]
EvelenLabs - Audio: [3:12]
Wav2Lip Model: [5:31]
Running it in Google Colab Noatebook: [9:00]
Thin plate spline motion model: [12:24]
Keywords:
AI avatar creation, MidJourney, Chat GPT, ElevenLabs, D-ID, AI tools, AI techniques, AI video creation, AI language model, AI voice-over, AI video platform
#chatgpt #midjourney #aianimation #chatgpt #aiart #elevenlabs 
Наука
Check out this video if you want another FREE technique: ruclips.net/video/7hfzoh_b1k4/видео.html
Does this other one show how to add some eye movements and blinks that make it look more natural?
Can talk to avatar live ? Any tutorial to do that without d ~id ?
No words can express my gratitude for the video.
Wow, thank you!
Thank you so much for the video. Great tutorial even for the course! For free is just beyond any expectations
Hi there I hope you don't mind me asking, I see you've used this..... Can I just ask is this something you can only use once. I have posed a couple of questions recently to the creator of this vid but as of yet, no reply. I am a novice and I am trying to understand about wav2lips. Thank you.
There used to be a great free app for Android called Speakpic, that did a passable job of animating a photo (with blinking eyes, motion, etc) and even allowed built-in text to speech. For quick results it was amazing. Sadly, it simply stopped working and the developer abandoned it.
I wish someone would release something else like Speakpic and keep it FREE.
Great tutorial. Much appreciated!
I'm also appreciated someone keep their works publicly, but again, would you work for free?
I love your video, I followed it from start to finish and created an amazing little talking video of einstein. Your a genius. Thankyou so much... The video certainly deserves a follow.
Thank you, glad to have you here!
Omg !!!! It worked 😄
I tried so many things,none of them worked
Great video, works as intended. Thanks man :)
Thanks :)
Thank you for the video. I have a question, do you have any other website or alternative to enhance the video avatar's quality?
Just wanted to say, your video's got us intrigued. Our tool also turns photos into talking videos, with a cool feature to customize the background. Ever tried something like that?
Very Cool! Thank you so very much!
Thank you, this was perfect. You delivered everything you said you would. It's all free 🙏🏿
Its not all free, Midjourney and 11 labs are not free, but then its a great tutorial
Very Informative Video Thanks Bro
Very cool & helpful. Would it be possible to have the avatar lipsynch and move with the audio & video from a filmed person in one go?
Thank you so much dear! 🥰💖
Fantastic!!
superb,,,great work.....Eye lid movement is necessary though for more accuracy and a natural feel.
Thanks, yes I agree with you. I have found a solution but testing it currently.
hope to see the video up soon!
@@engineerprompt Any updates on the eyelid movement?
@@classicalworkwolf2580 check out sadtalker :)
Thanks for the great video, nice job! If one was wanting to make a talking avatar of a real person, presumably you could just record a short video which could include blinking and real head movement so it looks more realistic and natural? Also, do you know if Wav2Lip can work in realtime i.e. so you can ask questions and have the avatar respond?
Yes, you could do the first part, there is another solution called SadTalker. Will be making a video on that soon. Its been due for a while. That will require a lot of compute to do.
Hi there I hope you don't mind me asking, I see you've used this..... Can I just ask is this something you can only use once. I have posed a couple of questions recently to the creator of this vid but as of yet, no reply. I am a novice and I am trying to understand about wav2lips. Thank you.
sure, waiting.
thanks for the tutorial , is it monetizable on youtube?
Athe downside is in DI-d's usage limit which requires credits. Is there another alternative to DI-d?
Hey mate amazing video and info, thanks! But do you know any a.I that animate the eyes ,like blinking and looking to the sides?
kaiber ai
thank you very much .
Welcome 😊
Thanks a lot for this, @Prompt Engineering. However, after it worked the first time, I'm now getting repeated "502 Bad Gateway" errors when clicking the "Interactive Demo" link.
First of all, Thank you for this amazing channel and great work.
I have followed your steps to create an animated IA Avatar, I managed to get the output of the video, but the result wasn't great at all because my Avatar was talking from his nose :)
Could you please guide me in how to fix this issue ?
lmao
@@stealthracerhd
Awesome! Are there any 'text to speech' tools that support more languages, such as Chinese or Japanese?
Yes . Fliki
Thanks a lot.
A video a week is mandatory, anything above that is a bonus
could me not clearing my cache and history be stopping images from processing. I used it once with no trouble, it was just what i am looking for.......now its doing nothing. am i over keen in pressing generate to soon ? I follow the instructions on the video and try to sync, but it just comes back with No File Chosen, any help pls. x
I get this Error... Anyone know why please? ValueError: --face argument must be a valid path to video/image file
is there any tool to make her a little more alive? like moving her head and eyes ?
Thanks
isn't it much more difficult work flow comparing to paid tools?
It's really great what you have here, but what about the eyes expression, they are too static. Could you think about a solution for those as well ?
Hi there I hope you don't mind me asking, I see you've used this..... Can I just ask is this something you can only use once. I have posed a couple of questions recently to the creator of this vid but as of yet, no reply. I am a novice and I am trying to understand about wav2lips. Thank you.
@@MedleyUnlimited Hi, I didn't tried this, I only asked about what was missing. I don't fully understand your questions: on 00:48 you have some steps to go for this, and it seems they are fully repeatable, of course, it's an iterative process which you can repeat as many times as you need. Hope I answered your question.
Awesome! Do you know a similar tool like wave2Lip, but to blink the eyes? From my point of view, this is the only missing piece to create a realistic avatar for free :) By the way, the MP4 generated by LeiaPix is not the quality than you. I tried with a couple of pictures on input, and the video is jerky a lot. Will try later to see it was a temporary issue on their side.
So I know one more solution; I haven't tested it myself but plan to do it soon. Email me and I would love to share that. For Leiapix, make sure that the image you use has depth information.
Hey check this out: huggingface.co/spaces/vinthony/SadTalker
@@engineerprompt I have carried out tests with SadTalker and the result has been good. On the fourth try I was able to generate a sample video. It worked for me when I used the 'preprocess:full' option, with the other two options I got an error message without further information. Thanks for sharing the link.
@@fernandodiaz8231 Nice, thanks for sharing your experience. I have yet to try it but its on my list to do. Will keep this in mind.
Hey mate amazing video and info, thanks! But do you know any a.I that animate the eyes ,like blinking and looking to the sides?
Can wav2lip be used commercially like on social media?
Can you please suggest other sites like leia pix to modify images? Thank you.
FINALLY something not freaking D-ID X_X, even if it doesnt work for my purposes, great to see someone trying something different..., do you know if it's possible to import the lip tool into a python so it can do the lip animation on the fly?
Yes, you can do that. I am running it in the Google colab, its python based :)
@@engineerprompt can you do a video on python and that tool?
@@engineerprompt can you do a video on python and that tool?
plssssss
followed the steps, but wav2lip keeps saying " didn't find any file withs format or mime compatible" any solution?
When I click to sunch the two in the interactive demo, all I receive is a message saying "Internal Server Error." I've tried it several times and keep getting the same message. However, in the collab version, it worked just fine.
After it worked the first time, I'm now getting repeated "502 Bad Gateway" errors when clicking the "Interactive Demo" link.
your step 5 is not working.... is their any other way to maintain video qualtiy?
Hey could you tell me AI generated video will monitize or not?
Does the Wav2Lip generation happen real time on code?
Great video! But I have a question: When I use the Thin plate spline motion model, the video comes out without audio. Is there anyway I can fix it?
In this case, you will have to add audio back in editors like Davinci Resolve etc. The model is only for motion transfer. So it removes the audio.
@@engineerprompt Got it, thanks
Thanks a lot for the video! I try, but I have like a blanck transparent square inside the face... And the resolution is very low. It depends of my gpu or absolutely not? Thanks for the help!
Are you trying a local installation or the web version? I haven't encountered this issue myself.
@@engineerprompt I'm trying the Google colab method like your tuto video. Is there a way to do this in local?? Thanks for your response!
@@76abbath Yes you can run it locally, checkout the github repo (github.com/Rudrabha/Wav2Lip). There are two sections you need to look at. First is the installation section (github.com/Rudrabha/Wav2Lip#prerequisites), you will need python and some other packages. Second is how to run it (github.com/Rudrabha/Wav2Lip#lip-syncing-videos-using-the-pre-trained-models-inference). You will also need to download their model checkpoints. Hope this helps.
@@engineerprompt thanks for your answer!
Thanks,, very interesting and I will try your method out, but the only prroblem is elenlabs is not free!, you only get 14 days on the free trial before you have to pay
Eye moments ? Any libraries you suggest for eye moments?
Check this out: ruclips.net/video/fDgQcDL-qOc/видео.html
sir, i follow all the system but step 3 , play result didn't work, help me so that i can
Please is there a way to remove the Avatar background?
Scary how she never blinks. :P
image lipsyncing is good but eyes are not blinking?how to add eye blinking or other facial expression in the image
You could do that probably with two pictures of your Avatar with open and closed eyes. You do the Animation with switching from Open to closed eyes in a Video editing Program. Maybe this works but its probably looking very simple.
bro only lips are moving...no eye bliinking or face movements are there like we get in D-ID..But at D-id we canot use custom audio like from seven labs...is there any good free d-id alternative?
check this out: ruclips.net/video/fDgQcDL-qOc/видео.html
What is the name of the converter
What oven temperature and for how long did you bake it?
About tree fiddy
fake it till you bake it
Great video! Is there a way to make her blink? ❤️🇦🇺
Is it not working anymore?
At 3.Play result video - 50% scaling part, then press "Play" I get this error:
FileNotFoundError Traceback (most recent call last)
in ()
2 from IPython.display import HTML
3 from base64 import b64encode
----> 4 mp4 = open('/content/Wav2Lip/results/result_voice.mp4','rb').read()
5 data_url = "data:video/mp4;base64," + b64encode(mp4).decode()
6 HTML(f"""
FileNotFoundError: [Errno 2] No such file or directory: '/content/Wav2Lip/results/result_voice.mp4'
did you try it in python?
No for Mac?
Hey good work, can you please guide for longer videos for free ?
Check this out. You want to run this on your local hardware:
ruclips.net/video/fDgQcDL-qOc/видео.html
How can I do it on iOS device
The problem is that the eyes do not blink , How can be set to blink ??? please help
check out SadTalker on github :)
hello, did you find a way to use one that looks more like d-id and with eye movements, but for free?
Yes, currently testing one. Have to figure out a few more things, video coming up soon!
Hey check this out: huggingface.co/spaces/vinthony/SadTalker
@@engineerprompt this is amazing. Almost as good as D-id. I found there were a few odd glitches in the video (the head and the neck just below the chin went out of sync in a couple of places, but the mouth and eyes were well done. Also the cropped version, removed the area below the neck, so it worked well. I don't know if they have a selection of voices, as I just used one I already had. Many thanks for this
You said you will have a video for the more natural avatar including head movement etc? Do you have it ?😊
Midjourney does not work free anymore, it tells me that demand is too high and I have to buy a subscription
can you help me create a talking cat avatar?
mine gives an error, do i have to select a wav audio file?
Yes, you need a wav file also make sure after each run, you change the name of the voice. I think by default it's set to Martin so on each run you will need to change that.
by doing colab , can we make larger videos of 5 minutes?
plz plz reply
Probably not, you need much more powerful GPU, it is memory intensive!
bro i want to animate aliant avatar like complex one how can i achieve this please help thanks v much
check this out: ruclips.net/video/fDgQcDL-qOc/видео.html
is this possible to python?
Yes, it's in Python
@@engineerprompt can u make a video doing this in python... pls
@@RonalRomeroVergel Yes, will do!
I've tried several deep learning implementations for talking avatars, including Wav2Lip, and they all have some sort of artefacts in the videos they produce (weird head movements, blur, glitches, etc). Not to mention how much time it takes for the GPU to run the models and generate the outputs. How do they create realistic avatars in a short time with seamless movements of the body, the lips and the eyes blinking, like in D-ID for example? I've been searching a lot, but I have not found a single tutorial talking about the creation process with open source tooling from scratch.
If it is not possible purely with deep learning, and if it turns out that I will have to use Blender or Unity or Unreal Engine or something like that, I am willing to take the time to learn. But even there, after I create a 3D model of the avatar, how do I then integrate it within a web app? I know how to do that for images and videos, but the 3D avatar is an object file, and its movements are going to be real-time and dependent on the text it is asked to speak. So, I am not clear on how the integration with a web app would work.
So, in essence, I just want to understand the step-by-step process of how to create realistic seamless talking avatars from open source tools without taking too long of a render time and with minimal artefacts. Do you have any tips on that?
D-ID is using their proprietary software so not sure how exactly its done. The best open-source tool I have come across so far (still testing) is this new paper: huggingface.co/spaces/vinthony/SadTalker
For improving the quality I think you need to do post processing on it. There are tools which can upscale videos just like images. Hope this is helpful.
@@engineerprompt Yes, post process the video using something like GFPGAN. Just split the video into frame, process the frames with GFPGAN to restore the faces, and stitch back together. You could also do this with Stable Diffusion. I like SD+Deforum as it will do all the splitting, frame interpolation, upscaling all in one spot. Can also use it to restore the faces. You could skip using the last step in this video if your goal was to improve quality.
I need one for 2863 Words for free please 🙏
This has a max of 20 seconds - !!!
why has mined got error in wavlip step 3. 10:50 minutes in time. How should i fix it can you help?
What's the error?
@@engineerprompt before the step 3, there an error in step 2. It says "Value error --face argument must be a valid path to video/image file')
same thing, any update?
@@animaltvstories5165 he doesn't care anymore
@@WeLoveMoralStories did you find the answer for this ? we are facing same error
Sorry I try it. Step 1 work, put in the files, step 2 work, upload files STEP 3 LIPS TO VIDEO - There is no file
Sory
Meow!!
:)
bhai code me error aa rahi hai
what's the error?
@@engineerprompt 2nd step : Create Wav2Lip video me value error aa raha hai
Value error: --face argument must be a valid path to video/image file
The excessive mouse clicking is very annoying & distracting. Otherwise, very nice job.
d-id is not free
Only 20 sec
If you run it locally, you can get a lot long videos.
@@engineerprompt what you mean? How?
@@egoldbeatz2619 You can download the repo on your computer and then run it there. If you have a powerful enough computer, you will be able to do it for longer videos.
Wasted my time 😢
Thanks
Welcome 😊