How To Make A.I. Animations with AnimateDiff + A1111 | FULL TUTORIAL
HTML-код
- Опубликовано: 27 май 2024
- *THERE IS NO NEED TO DOWNLOAD THE FORKED EXTENSIONS ANYMORE, THE NATIVE ANIMATEDIFF & CONTROLNET WORK TOGETHER AGAIN. The links below have been changed to the NATIVE extensions; install them.*
In this tutorial, I will show you how to install ControlNet & AnimateDiff in Automatic1111 so you can animate images without getting the "AttributeError: IPAdapter". I will show you the best settings to use, the ideal workflow, & show you how to increase the quality of your animations by interpolating the frames to 60FPS+
👑 BECOME A TYRANT & CONQUER YOUR LIFE:
👑 Join the Tyrant Empire: tyrantempire.com/register/tyr...
👑 Conquer A.I. Art: tyrantempire.com/prompt-gener... USE CODE: " RUclips25 " FOR 25% OFF!
🖥️*VIDEO LINKS*
1 - AnimateDiff: github.com/continue-revolutio...
2 - ControlNet: github.com/Mikubill/sd-webui-...
3 - Motion Models: huggingface.co/guoyww/animate...
4 - ControlNet Models huggingface.co/lllyasviel/Con...
5 - TOPAZ VIDEO AI: www.topazlabs.com/topaz-video...
⛓️*SOCIAL LINKS:*
Instagram: / tyrinthetyrant
Twitter: / tythetyrant
Soundcloud: / rampayj
⏳*TIMESTAMPS*
00:00 | Introduction
01:00 | Installing Fixed Controlnet
01:29 | Installing Fixed AnimateDiff
01:44 | Disable Original Extensions
02:12 | Installing Motion Model For AnimateDiff
02:41 | Installing Controlnet Models
03:18 | Ready To Animate
03:33 | Text to Video
04:16 | How To Fix GIFs That Change
04:40 | Enable This Setting
05:00 | Image To Video
06:00 | Image To Image To Video
07:18 | TypeError: VanillaTemporalModule.forward()
07:53 | Fine-Tune Our Prompt/Animation
09:17 | Improve Quality: Upscale & Interpolate
11:50 | Conclusion
🏷️*TAGS*
- Automatic1111
- AttributeError: IPadapter
- AnimateDiff
#️⃣*HASHTAGS*
#stablediffusion
#AnimateDiff
#aianimation 
Хобби
THERE IS NO NEED TO DOWNLOAD THE FORKED EXTENSIONS ANYMORE, THE NATIVE ANIMATEDIFF & CONTROLNET WORK TOGETHER AGAIN!
New Technique Showcasing The Prompt Travel Feature In AnimateDiff Here: ruclips.net/video/gLxQEyaDBt8/видео.html
so where do we paste the model we download, which folder?
Thanks for sharing the update on the Controlnet fix. Just to let you know to do transition, prompt travel is a new feature added to AnimateDiff WebUI version and can now do transition of different poses seamlessly too.
you are the only channel that explains why the animation changes half-way with something different. I went crazy, I had no idea how to fix it, thank you!
Subbed because you were the only tutorial I got animate diff to work.
Thanks for the tutorial mate! You really helped me out here☺️
Best tutorial I have seen in my entire life
thanks for this!, I'm glad I found your channel
Great tutorial, you saved me a lot of time troubleshooting! Thanks!😀
Perfect tutorial. Not too fast or slow. 👌
When disabling sd-webui-controlnet and sd-webui-animatediff, I no longer have the option of seeing the Animatediff tab at the bottom? I installed the two other things sd-webui-controlnet-animatediff and sd-webui-animatediff-for-ControlNet. Why doesn't the animatediff tab show up? :(
I haven't had much luck with getting AnimDiff to work in the past. Will try this.
Hello, I followed your video tutorial to use ControlNet for converting images to GIF. The generated GIF animation is not very pronounced; only the hair has a slight movement, and the overall picture is almost indistinguishable from the original. What could be the reason for this?
animate diff is not showing in my UI, only the controlnet for animatediff...anyone had this issue?
Hey Bro.. Looked long for a Animate Diff Tut that I like, with a lot of Information and a pretty good guide.. Thank you 💪🏻😎
Curious if the "AnimateDiff v1.6.0 for Controlnet" won't show up for anyone else. I tried reinstalling both (Anidiff 1.6.0 and CtrlNet v1.4.0 AniDiff) of them multiple times, deselecting other extensions, disabling original Anidiff and controlnet, restarting cmd and pc. Only CtrlNet v1.4.0 AniDiff shows up.
same for me as well
When i use AnimateDiff, i just get a grid of images and a bunch of separate images, what am i doing wrong?
Same here
@GeekRedux I sort of figured it out. I changed my checkpoints from SDXL back to v1.5 and it worked
@@DarkFlarePrince Same thing for me this morning. Noticed the "15" in the motion module name and was like, "Hold on". Glad it's working for you, too.
apperciate it!!!!
Great work! Have you had any success with running this through sdxl?
Thanks was getting errors using the available extensions with animatediff. Do we need Xformers installed though for it to work properly?
For me it doesn't generate a video/gif/frames... It just does as it always does, generates individual still frames. I have done everything in the video and everything looks the same/seems fine. Does anyone know what is wrong? I am usin SD 1.5
you may need to upgrade xformers, pip and pytorch. AnimateDiff should work on SD 1.5 but if you have been running older versions of SD previously like me some components may need updating
Is Tile/Blur preprocessor specifically designed for V2V? What exactly does it do?
Como optimizaste la interfaz bro? No me deja y tengo 8 VRAM igual que vos ayuda?
is there a way to do this without the tile controlnet so that it works with the sdxl model or any xl model.
Video url Link and discription link is different, why?
Thanks for your video! It's amazing how fast the 2-second video is generated. It's taking me 7 minutes to do so on Mac Mini Pro with 16 GB of memory. Can you share your device memory specs? If the length goes beyond 3 sec, MPS error occurs so I am so frustrated. Thanks in advance.!!
i don't see that first link you mention in your description🤔
Great video bro! by the way, could you explain the video transition with more detail? which you showed the previous video
Thank you.
For the transition, you put the last frame of the animation you made into the 1st controlnet. Then, if you want to transition that animation into the next, put the first frame of the animation you want to transition to into the 2nd controlnet.
So when you generate, you are creating an animation that ends at the first GIF & leads into the 2nd GIF!
Hopefully that makes sense.
Join the Tyrant Empire if you want to DM there as well.
i still dont know why this method can work.I try to use two tile controlnet to control.but the GIF still merge two tile image together,haven't seen the transition result.@@TyrinTheTyrant
i guess id better look up txttovideo because i dont see that option on my auto1111
Thanks for the tutorial. The image to image one doesn't work for me. For me it seems like it merges both pictures and then I only get a flickering animation.
I followed every step but I when try to use Tile/Blur to anime it takes waaaay too long to generate like 1-2 hours for 2 seconds gif. When I use other option it only takes a couple minutes. Why does this happen?
When i add an image to control net ..... my video no longer animates ..... it just animates a still of the image i put in controlnet.
Ive tried like 50 times and checked over everything ..... any clues ?
When I use controlnet with animatediff controlnet is applied on every single frame, not just the first one. Which removed any animation whatsoever and just makes a static video.
Hello, pls did it ever happen that the AnimateDiff for ControlNet tab wont show for you? I followed every step, installed all the stuff but for some reason, the tab stays invisible. I did the settings, to hide and show only the stuff you suggested. But still the tab is not visible. Pls any suggestion on what I could do?
about the red to white, what if i want to go from red to green, green to blue, blue to gray and then gray to white? how to achieve this?
any recommendations on converting it to a gif so it loops?.
I'm getting 20 individual images that match the prompt, not a 2 second animation of 10 frames per second. What is the problem?
Every time I try to generate a gif it gives me the error: Controlnet object has no attribute 'label_emb', what could that mean?
Unfortunately it doesn't work with an 8GB GPU at all.
Thanks so much for your tutorials! Any possiblity of doing Vid2Vid with Automatic1111?
That is on the list of things to explore.
I will post a video as soon as I find a quality method that works 👍🏽
Awesome! Looking forward to it :)@@TyrinTheTyrant
Can't see the AnimateDiff tab :( Made sure the others were disabled, everything is updated, restarted everything ... nada!
I tried your extensions and just got a bunch of errors. Experimenting with best settings for regular versions now.
One of these extensions refuses to show up in my UI. I have the ControlNet v1.1.410 for AnimateDiff showing but not the AnimateDiff 1.6 for controlnet menu, it's missing.
Nevermind, i realized my git pull was not updating properly, and it didn't know what branch i was on, so i had to run "git checkout master" first, then git pull, then it updated to SD version 1.6 correctly, and the extensions now work fine.
Great tutorial.
Unfortunately, I'm getting an einops.EinopsError: Error while processing rearrange-reduction pattern "(b f) d c -> (b d) f c".
Input tensor shape: torch.Size([2, 4096, 320]). Additional info: {'f': 16}.
Shape mismatch, can't divide axis of length 2 in chunks of 16
Any ideas on how to fix?
same also stuck on this error
Hi @@ThoughtsFew
I fixed this error by updating Stable-diffusion.
To update add this line to your Webui-user.bat between the set commandline_args & the Call webui.bat:
git pull
Unfortinately, now I'm getting the error: torch.cuda.outofmemoryerror
Thank you! The update fixed this error for me. Now I'm stuck with black squares due to the NansException error. I have tried the --no half vae command line arguement solutions but combination has worked. I'm assuming maybe I need to update torch to 2.1.2 :p@@user-db6fl8fw9t
I haven't encountered this error.... yet. If I do i'll let ya know if i find something.@@user-db6fl8fw9t
Decreasing the resolution fixed this error
The integration of ControlNet and AnimatedDiff is awesome, though this has such a high failure rate. It crashes every 10 gens, or I just get colorful static. Being limited back down to 32 frames isn't great. Per notes that isn't going to get updated anytime soon. I think I might just have to learn ComfyUI to get around this.
Seems you have another issue, works flawlessly here
prompt padding will fix the prompt length issue
Alright beginner question. I'm limited to 10GB of VRAM. Is Animatediff just a no-go without 12GB?
Not necessarily. With A1111, I've heard of people getting AnimateDiff to work with 8GB.
Worst case scenario, you can use ComfyUI, I've heard you need a minimum of 5GB.
Try passing the argument "--medvram" in your webui batch file if you are running into CUDA errors
Hi, iv followed all ur steps and seem to have checked all the settings you have enabled but for some reason its still only generating images, not gifs
same here
Same
Because this tutorial was bad and it will break your build, you may now have to delete your VENV folder and let it reinstall because you will continue to get errors.
This tutorial was a quick fix for an update that made AnimateDiff & Controlnet unusable.
They have since been updated & now work together.
There is no need for the forked extensions anymore.
Hey I followed everything and when I generate it gives me a grid image, all the individual frames ( they're not coherent for some reason) and it never generates a gif... I can't get it to work. I searched everywhere. does anyone have an idea on how to fix that?
You are sure the motion model & animatediff extension are enabled correctly?
I had this issue as well, kept getting a grid output in txt2vid/txt2gif while using photon_v1 as the checkpoint. No issues with the img2vid/img2gif though... pops out a gif like no problemo. So easy work around for me was to generate initial pic in txt2img then send it to img2img to do img2vid
Followed directions. I get this
RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. (disabling xformers fixes it but its 45 min generation time for your first example. )
Try passing these in your webui file:
set CUDA_LAUNCH_BLOCKING=1
set TORCH_USE_CUDA_DSA=1
@@TyrinTheTyrant ER....Thanks for the help... After trying that, It gives me
RuntimeError: CUDA error: invalid configuration argument Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
(I'm about done trying tho xD... normal image generation works fine for me... I mostly do SDXL 90 seconds an image. ((sd1.5 images are like 10 seconds))
hi! nice video! I also had this error and your suggestion sounds in the right direction, but thats just safety and error measuring , not doing anything different . I had some good results when removing xformers. I'm on a 1080ti with 11gb vram , updated SD and torch and xformers, and disabled all extensions @@TyrinTheTyrant
@@madwurmz Ah, i'll keep that in mind. Thank you for the info!
i have the same issue,did you find the solution?@@AtheistExpert
Just outputting a png.. no mp4, gif...
Errors on command line
AttributeError: 'AnimateDiffProcess' object has no attribute 'text_cond'
AttributeError: 'AnimateDiffProcess' object has no attribute 'text_cond'
AttributeError: 'AnimateDiffProcess' object has no attribute 'text_cond'
repeats while building .... Ends with:
AttributeError: 'NoneType' object has no attribute 'save_infotext_txt'
Hi, even following the video, I only get 1 photo, no video and no gif and I don't understand why :(
Hmm, that could be a number of things.
Send me screenshots of your U.I. in the Discord!
Installing all of that stuff broke my a1111. I had to delete the folders to get it working again.
Definitely hit or miss.
New animatediff update works with the native controlnet now, so there is no need for these forked versions anymore.
Can you elaborate on that? OR rather, simplify that down so an idiot can understand? Do i no longer need the two extensions you recommended in this video? What do i install instead? For me, i'm not even getting the two animatediff for control and controlnet for animatediff options in my UI, i only have the standard controlnet options, which is weird, because i disabled that in the extensions, like you said to. I don't know what's going on. @@TyrinTheTyrant
Same, trashed my whole build. Not cool man
@@wholeness Nevermind, i realized my git pull was not updating properly, and it didn't know what branch i was on, so i had to run "git checkout master" first, then git pull, then it updated to SD version 1.6 correctly, and the extensions now work fine.
When I click generate nothing happens
So strange....It just flat out does not function when I do these exact steps. I always get an error about missing keys and then it just generates still frames and no gif. But also the updated extension doesn't even show up if you remove the old extension instead of just disabling it. I appreciate your tutorial, but it seems like animatediff is very temperamental and refuses to work for several of us.
The original extension works perfectly with controlnet now, so there is no need to use the forked extensions.
Have you tried downloading from here?: github.com/continue-revolution/sd-webui-animatediff
@TyrinTheTyrant I've tried both, and just can't get it working. Same error on both extensions. I'll mess with it more later.
if i could give a recommendation, comfyui might be more stable in terms of breaking changes. with that, it is plug-and-play with all kinds of different functionalities (animations/pics/upscale/controlnet/etc), and you can copy in and use complex graphs without knowing how to set them up. it is not as hard as some of the big graphs make it look, basic graphs only need 4 nodes or so.
Comfyui sucks
animate diff extension got an update
does this work on rtx 3050 8gbvram?
i hope so?
doesn't run on my 8gb GPU
does this work on 8gbvram 2070 ?
will need the --medvram argument and even then it will take a bit to generate 512x512, dont even think about going higher. the comfyui version should work quite a bit better though as it's a lot easier on your gpu. I've used both versions and the comfyui version is the more efficient and more stable version. I had a 2070 8gb but recently upgraded to a 3080 10gb and i can barely run this in A1111. Hope this helps :)
@@ZertyAI yes, Not ready for 20 series i seems, but it is ok - it will be optimized
using a 6800XT on Linux is a rough experience lol
why is it my generation looks nothing like yours (txt2img)
All this time for a poor result like a morphing made on Windows 95.
AnimatedDiff does not work! It uses too much memory! Not enough memory!
You need at least GTX 3090. Is that what you have?
so GTX 3050 8gbvram not gonna cut it?@@AndyHTu
@@x_XZeroX_x you might be able to use some of it but its too low memory and will give him errors.
You*
@@AndyHTu You mean RTX 3090? Yeah, I don't have that. Also, if I were to get one, is it possible to replace the GTX 1660 SUPER that I have in my gaming desktop with an RTX 3090 without losing anything on computer. I ain't paying thousands of bucks on a whole new computer just for that one part that I need, unless it's a gaming laptop.
As a man with a "little" hearing problems, i find the music you use is really disturbing, so ended up just muting you and read the sub, so plz plz, for the love of the worms, stop playing music :)
This guys is dyslexic, it was so confusing, he has the wrong links and does not do a good job of explaining things....
True, kind of funny watching him copy the DavideAlidosi branch and gives link to thiswinex one.. no wonder the how many " doesn't work" comments lmao
P.S. All of you should just switch to forge ver...
worst green screen ever,lmfao
- I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it is just our flesh and that is it. It knows only things of the flesh (our fleshly desires) and cannot comprehend things of the spirit such as peace of heart (which comes from obeying God's Word). Whereas we are a spirit and we have a soul but live in the body (in the flesh). When you go to bed it is your flesh that sleeps but your spirit never sleeps (otherwise you have died physically) that is why you have dreams. More so, true love that endures and last is a thing of the heart (when I say 'heart', I mean 'spirit'). But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons and love for selfish reasons that is a thing of our flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. Take note, love works in conjunction with other spiritual forces such as faith and patience. We should let the Word of God be the standard of our lives not AI. If not, God will let us face AI on our own and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. Our prove text is taken from the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21. Let us watch and pray... God bless you as you share this message to others.
*Just remember a god that be killed by his creation isn't god*
Nice tools and way to produce videos and images. I am looking for a way to produce sprites sheets with Automatic 1111 and controlnet if needed, to have a consistent character moving or doing other actions to use in other softwares. That would be good