NEW AnimateAnyone LOCALLY. MIND BLOWN!
HTML-код
- Опубликовано: 28 май 2024
- 1 click animations with consistency. Now locally in Stable Diffusion.
Patreon workflow / animateanyone-97409128
Local install + simple free workflow github.com/MrForExample/Comfy...
HF Demo huggingface.co/spaces/xunsong...
Install Comfy and Manager • How to install and use...
!!!
A lot of users seem to have issues with conflicting nodes. I would recommend installing this on a fresh Comfy to not break your old one (unless you know what you're doing). The fix would be to just remove the newly installed custom nodes again.
Some users have also reported that updating their pytorch to match their cuda version has helped. Here's how a user with cuda 11.8 solved his issue:
pip install torch torchvision torchaudio --extra-index-url download.pytorch.org/whl/cu118
If you have errors from missing custom nodes. Try a fresh install of Comfy, make sure it's the latest version. You can also try manually installing AnimateAnyone (can try git clone from github into custom nodes folder).
Prompt styles for Stable diffusion Automatic1111, ComfyUI & Vlad/SD.Next: / sebs-hilis-79649068
ComfyUI workflow for 1.5 models: / comfyui-1-5-86145057
ComfyUI Workflow for SDXL: / comfyui-workflow-86104919
Get early access to videos and help me, support me on Patreon / sebastiankamph
Chat with me in our community discord: / discord
Stable Diffusion for Beginners Playlist • Stable Diffusion Begin...
My Weekly AI Art Challenges • Let's AI Paint - Weekl...
My Stable diffusion workflow to Perfect Images • Revealing my Workflow ...
ControlNet tutorial and install guide • NEW ControlNet for Sta...
Famous Scenes Remade by ControlNet AI • Famous Scenes Remade b...
CHAPTERS
0:00 This is AnimateAnyone
0:45 Huggingface demo
01:29 How to install AnimateAnyone Local
7:16 How to use AnimateAnyone 
Хобби
OMG I was waiting for this for at least a year, let's go!
Today's the day!
there are some mismatches recently if you try to install the things in the order of the video, after installing the custom nodes you will see the errors you see in the rest of comments, you have to manually upgrade/install some libraries: pip install --upgrade diffusers and ffmpy library if you see its error in the console, same as any other errors, leaving this here for the next person
This no longer works because there is something missing from on of the requirements files so you end up with
When loading the graph, the following node types were not found:
[ComfyUI-3D] Animate Anyone Sampler
[ComfyUI-3D] Load UNet2D ConditionModel
[ComfyUI-3D] Load UNet3D ConditionModel
[ComfyUI-3D] Pose Guider Encode
[ComfyUI-3D] Load Pose Guider
DWPreprocessor
Nodes that have failed to load will show as red on the graph.
also when reported on the github they closed it withought fixing it
they don't care, it is all single use. They just want to shine with some new feature, let some people reproduce it on the day of release, and when the next guy publishes something he breaks everything else
it works for me
Awesome,
Thanks for sharing.
Yo great video :) thanks for your work! If I can ask from where are you getting so smooth and clean only open pose videos to feed it?
incredibly useful tutorial. thank you so much id be lost with it
Glad you liked it :)
Wow!! Thanks for showing us 👍
My pleasure 😊
nice and it gets better
hey sebastian! thanks for the video, but i can't get it to work. i get a (IMPORT FAILED) ComfyUI-3D-Pack ERROR in the manager. reinstalling the node didnt work. any ideas?
I’m with this error too, wished I knew which python he used
same here
Yes! Been waiting for this to drop!
I have an error that says ModulenotFoundError no module named "diffusers" i've tried to pip install it but no luck :/
Just pip upgrade diffusers
do i need to download every file in the hugginface repos
I can't get the Custom Node to install correctly. Not a clean Comfy Install (I want this on my main), Git Cloned, and missing the 3-D nodes, and the ComyUI-AnimateAnyone-Evolved gets (IMPORT FAILED). Any ideas?
The " comfyui-3d-pack " is missing the " diff_gaussian_rasterization " module from its pip requirements. I'm assuming the author has that already installed on their machine but it wasn't added to the " comfyui-3d-pack " python requirements. Just about to try installing the submodule directly from the INRIA " gaussian splatting " repo ..
how do you install the submodule directly? I want to try that but that sounds like a foreign language lol what are the exacts steps? Did it work for you?@@ctankep
@@ctankep Did you get it to work? I'm still stuck on this one.
I got the same issue... 3D Pack not able to be imported while boot up... If you find a solution please keep me updated :)@@Foolsjoker
so, clean install of comfy, git cloned the folder and installed the requirements, but loading the demo file from animateAnyone, I get a lot of missing stuff.. like ComfyUI-3D and VHS_videocombine - and I dont have the manager tab on my comfy.
The manager needs to be installed manually github.com/ltdrdata/ComfyUI-Manager
When did they release it? I didnt think it was released yet? I thought they released magic animate but animate anyone was still in development?
and how to install cuda 118 when I am using a windows portable version? there is no CLI to place it
Hello, can you tell me what CPU and GPU your computer uses to run these models?
i am getting torch not compiled with cuda error, how to solve it?
So did I. I tried almost every solution on RUclips and online. None worked. The only thing that worked was a clean re-installation of ComfyUI. And then the "torch not compiled with cuda error" re-occurred when installing the missing nodes. So I could not proceed with the free workflow.
Is there some community resource where I can find various animation skeletons? I'm looking for game stuff like top down idle, running, attacking and stuff like that, or side view of the same. I imagine a pipeline for making game character resources.
I did a video on a free online tool like this a while back... I just can't remember the name 😅
@@sebastiankamph I got pose estimator working so now I’m looking for consistent videos of people moving in specific ways from various angles too. Use case is generating sprite sheets for games, but there are few free resources for good mocap work in my search results.
Still getting these messages and several red boxes. Did anyone solve this? When loading the graph, the following node types were not found:
[ComfyUI-3D] Load Pose Guider
[ComfyUI-3D] Load UNet2D ConditionModel
[ComfyUI-3D] Load UNet3D ConditionModel
[ComfyUI-3D] Pose Guider Encode
[ComfyUI-3D] Animate Anyone Sampler
Nodes that have failed to load will show as red on the graph.
yep, lost hours of my life trying to get past it, nothing yet
I'm now hours in as well. Even tried the torch suggestion. Did nothing for me.@@jonescrusher1
@Skydam33hoezee @jonescrusher1 If you're getting that error because ComfyUI-AnimateAnyone-Evoled couldn't be imported, then you can try this workaround:
1. Open the manager
2. Go to "Install PIP packages"
3. Type "diffusers" and click "OK"
4. Restart ComfyUI and it should be fixed
Thank you for the tip, but unfortunately no progress. Same errors. @@dovasubs7212
Hero.@@dovasubs7212
Does this do creatures as well or only humans? Also, would it be possible to just do the character against a green backdrop?
I totally felt that 11:50 espresso moment :D
Omnomnom
I dont get how to use comfy ui , can't we use this in stable diffusion?
Can one make prawn videos with this tech? Just asking for a prawn director friend.
🤣
Does anyone know if there is a website somewhere that offers free open pose animated mp4's for download? I would love to play around with these animation nodes, but I don't really have any decent videos to base them on.
Pexels -> DWPose(openpose, see video)
@@sebastiankamph Excellent resource! Thank you
thank you ! I cannot install it right... it gives me this error "The torch environment may be compromised due to version issues as some torch-related packages are being reinstalled." do you have any Idea?
it works when reinstalling a new fresh comfyui..
Glad to hear you got it working
this is not animate anyone , the code is not yet released and achieves much better results. this is moore animate anyone, a project which aims to achive similar results but doesnt (yet). its still good though, ill try it with a raw drawing and an action scene made in blender. hope it doesnt just work with flat tiktok style dance vids but with camera and perpective
How's the result ?
All red boxes are gone, but when trying render the Animate Anyone Sampler node gets a purple border, and this error message appears: Error occurred when executing [AnimateAnyone] Animate Anyone Sampler:
mat1 and mat2 shapes cannot be multiplied (2x1024 and 768x320)
To try and troubleshoot it, I only use input from the example workflow... What could be wrong?
Same error here...
Great tutorial!! Is it ok to rename the models? Thanks in advance!
Sure, go ahead. Just need to reselect them in the workflow.
hi @sebastian have u removed the work flow from your patron.. i was going to purchase today..
No, should still be there :)
How many frames can you render at the standard resolution for this model (512x768)? My RTX4070 12GB - 20 frames
I'm returning an import failed error when comfyui tries to load custom nodes, the comfyui 3d pack isn't recognised even though it's been downloaded, ModuleNotFoundError: No module named 'diff_gaussian_rasterization'
hi, thanks for the video... i tried installing the node but am getting " (IMPORT FAILED) ComfyUI-AnimateAnyone-Evolved" - i tried both manager and direct pip install... neither is working :( any ideas... ?
In my case in had to disable AnimateDiff Evolved in Manager
Good stuff as always mate. More toys for me to get less real work done. 😢
Hah, you got that right!
i want to try Magic Animate,... but im gonna try this as well
magic animate sucks unless you prepare a 3d character with exact proportions of your final character and use that as input
I downloaded the custom node, and now comfy ui does not want to work.
Yup my Comfy is broken too after installing this... what a waste of time.
Same for me. I tried almost every solution on RUclips and online. None worked. The only thing that worked was a clean re-installation of ComfyUI. And then the "torch not compiled with cuda error" re-occurred when installing the missing nodes. So I could not proceed with the free workflow.
Hi, how can i get the openpose version of my video that i want to use?
Just run it through an open pose preprocessor and connect a VHS combine node to save the output as video if you watch the video it's in there too maybe not explicitly explained but.
Yo I got another one for your intro :
What happens when you kill a board game?
It dice.
Oh, I like it!
can I use in google colab or Kaggle also ?? I have installed and used comfy ui in kaggle
Phenomenal video as always. I hope you get some well deserved rest!
Thank you kindly! :)
Can you do this on the online platform as well or do you need to have the tool locally installed?
You can do this online too on ThinkDiffusion.com
HI I use comfy ui in kaggol can a use animate anyone there tool with your all workflow ?
This is my new favorite channel... dare I say this channel is perfect?
what is the maximum number of frames it can process at a time ? i tried animatedif before and for my gpu it was limited to 16 frames
I didn't try over 300 at this time. But I had no problems with it so far.
thanks Sebastian, how much v-ram do you have and what gpu are you using
I also get capped at around 16 before running into a memory error. Is there a way to dedicate more V ram to comfy in settings? How are you getting 300 frames? I’m on. Ryzen 9 + 4060 with 32 gigs of memory.
I’m also wondering which node parameters will decrease memory usage in addition to the load video height and width. Something in the K sampler maybe?
Any tips on maximize my output frames / mitigate render times would be hugely appreciated.
@@calvinherbst304 its a v-ram issue, I believe he is using a 24gb card or more
@herbst304 on my 3090 it uses 13.5gb vram for 270+ frames
I was moved by your wonderful lecture. I'm trying this too, but I'm getting the (IMPORT FAILED) ComfyUI-AnimateAnyone-Evolved error several times. Is there a solution?
Conflicting nodes, try fresh install.
@@sebastiankamph thank you for telling me!
Fresh install didn't help :(
Idea... what about having a green screen solid color behind character as a way to avoid the jitter problem?
Try it and let me know!
Minimum VRAM requirements?
Excuse me, is there something similar in automatic1111?
can we get them to blink also?
I get two deep red nodes from the Patreon workflow, and the manager doesn't seem to want to find them. They're for image height and image width.
and getting "UNKNOWN ERROR" every time I try to run it. Don't know what I'm doing wrong
You need to install AnimateAnyone manually. Sift through the comments though, a lot of people seem to be having conflicting nodes. (Try fresh install of Comfy)
I will, thank you @@sebastiankamph
How long does it take to render on a 8gb gpu?
It'll probably be faster on a 4060Ti than on a 1070
Is this only available for SD1.5 and not SDXL? 😢
Will my 11gb gpu be able to run this?
I downloaded this animate node through the manager and it broke my ComfyUI. Broken meaning got a pause press any key in the command prompt window and comfy wouldn't come up in browser it just quits after pressing any key. I have a Comfy portable install. AssertionError: Torch not compiled with CUDA enabled
same problem fixed it by updating via the comfyui update folder updated python dependencies and comfy
@@flavorbot I tried that and I still couldn't start it again so I had to just do another install. I don't know why. I might try it again but I'll do another separate clean instance so if it kills it again I won't care. Live and learn.😁
Impressive what AI can come up with. Hopefully tiktok won't copyright their dance moves lol
That's only a problem if you wanna make your work public.
Not Able to Subscribe to Patreon, every card payment from India seems to be failing.. Any Idea whether this is a Patreon issue?
Hmm, no idea. I've never heard about that before. If you're looking for anything specific you can dm me on Discord. But I can't add users in Patreon.
Also valid for Türkiye. sad
Ahhhhgh blast man as soon as see Nodes im out ha
Hey Seb!
Its barely been a month and the AI industry is already rolling like crazy...
What are your predictions for 2024! 😁
Infinite supply equals zero demand. So I predict total collapse is imminent.
Some nodes stay red no matter what unfortunately.
can u show a tutorial with sd1.5
This is 1.5
@@sebastiankamph I was gonna say~
Also, shame on you for dusting off that tissue joke 😆
my bad i probably shouldve asked if there is one for A1111, @@sebastiankamph
How can we do this in Think Diffusion? :B
Same way I showed you here. It's like using your own local machine. Just drop the models in the correlating folders in the file manager.
boogie woogie. big fat cheers!
Thanks Ronny! Good seeing you here 🌟
i tried installing animate anyone and broke comfyui (Torch not compiled with CUDA enabled) please hel fix anyone that knows whats wrong i tried a fresh install but that produced teh same result
With the fresh install, try manually installing animateanyone first through a git clone, then the others from the manager.
This happened to me as well, but creating a fresh install of Comfy seems like an extreme solution given all the nodes that I have installed at this point. I'd have to redownload every custom node I've ever downloaded? Maybe I'm just not understanding.... Is there no way to undo this so that I can run comfy again? Or is my current install with all its nodes useless now? Just trying to understand what to do to get back where I was before installing this.
You could just remove the newly installed custom nodes. This is a common thing with conflicting nodes.
@@BobDoyleMedia
@@sebastiankamph my solution ended up being reinstalling pytorch to match the Cuda installation I had. Somehow going through this process, changed something somewhere, but updating PyTorch appropriately fixed everything. No fresh install required!
Same as @BobDoyleMedia, after installation it broke my ComfyUI 😞. Removing of the newly installed custom nodes didn't help. Reinstalling pytorch helped, but now it shows a windows message "python entry point not found". SOmething is still wrong. Arrrgggghhhh 😞
EDIT: I had to completely reinstall ComfyUI, moved all nodes + models from the old one, fixed some nodes which neded some requirements/dependencies.
we're not that far from being able to make movies at home if we can animate characters like that. Now you just need to animate a blender rig and the render will be done in comfyUI ! No texturing ! x)
You could automate to that level now, the problem is doing it that way without it being obvious, and we aren't really close to the point where it won't always be obvious. It isn't just jitter, look at the frames with a messed up face, look at how the sun moves like it's painted onto a backdrop. There's some seriously disconcerting elements of this that won't be easy to avoid using automated techniques, or fix at all without resorting to texturing.
@@michaelleue7594 There are other workflows that take care of these things. The main use of this kind of workflow would be to rip animation from video onto a base video which can then be masked and composited in other ways and style animated in other workflows.
@@aaagaming2023 I'm curious how work intensive those workflows are compared to just texturing from the beginning. I don't know of any that produce a significant savings without sacrificing a lot of stylistic control.
@@michaelleue7594 Its more a difference between micro tweaking a few settings and going through randomized seed generations of just a few frames at a time, as opposed to knowing what youre after and manually texturing. But the workflows to do pretty much everything are already there. Masking and compositing are one of the big strengths of comfy.
how much Vram required for this process?
how to make motion captures?
Shown in the video with openpose from dwpose estimator.
thanks!@@sebastiankamph I must've missed that - was watching over lunch
Hello fellow starcraft fan ! Thanks for the tutorial !
Oh why helloooo! What gave me away!? 😅😍
Saw you in Harstem's chat the other day x) ^^ @@sebastiankamph
@@jimdelsol1941 best SC streamer by far! Watched way too much
With good timing, a snotty-nosed dad-joke can be a runway for a pretty fluid animation.
Reading the comments discourages me from trying it. Not keen on installing comfy again just to end up stuck because some stuff is missing. Its 2024. Not doing that shit this year. Maybe next.
man i really wanted to try this but i got the error as well no i wish they would fix this
There are some new troubleshooting tips in the description
I notice when u try to animate real photos the results are worst than when you animate anime or digital paint characters stuff, for example I notice that background was going to fuse with the legs or other stuff belong to the character
lil bro actually wrote his name on his hat in CRAYON… 😭
Dad energy
Could you share your workflow, perhaps on your Patreon? Just kidding! 😂😂😂 Seriously though, great video, Seb. However, mentioning your Patreon five times does come across as a bit much for the viewer.
You know I need those expensive steaks for breakfast, lunch and dinner to get me through all that dad joke work 🥰
hello so can anyone help me
I had this message when I want to queue prompt
"Prompt outputs failed validation
VAELoader:
- Value not in list: vae_name: 'sd-vae-ft-mse.safetensors' not in ['taesd', 'taesdxl'"
I don't quite understand what it means. I put the vae where it should have been but not in taesd?
Thank you
Has anyone used this for anything other than front facing dancing TikTok girls? I've literally seen no other videos made with it that shows anything else. I'm thinking it's probably because it's useless at anything else. Even the front facing dances often have issues with limbs passing through other things, plus it's a long way from being consistent. I'm thinking as soon as you tried to do anything useful and more complex with it that involved changing camera angles or body rotation or something like a side shot of someone walking, it would fail miserably.
Much better than animation used to be but still a long way to go before it's more than a gimmick to make dancing anime girls.
It's mostly trained on static camera stuff, yes. And I guess in regard to what you see on TikTok, that's what people interact with and what's shown. I'm sure there are other/better examples that aren't shown as people interact less with it.
SD video sucks, this is just the latest masturbatory tool of the week so youtubers can upload something, and anime fans to generate more tiktok trash, that's about it.
no automatic1111??
Not yet. At a later date I guess :)
Why so many dislikes?
No one likes comfy
They get triggered by ComfyUI 😂
Im really going to need to start looking into Comfy huh? Haha
I'm afraid all the new tech gets released to Comfy first. I'm sure there will be an a1111 extension soon.
@@sebastiankamph How is Comfy with AMD GPUs? I've been having issues for a while with my AMD card when it comes to AI generated content.
@@khroomlet8821 amd sucks for creative software , just get that through your head and make a better purchase next time
It 'breaks' Torch...
Comfy UI only? Why not Auto1111?
Not available in auto1111 yet.
Lets say it clear - if you install it you probably will break your Comfy installation, just a warning.
Chapters?
Added!
Means a lot. Thank you! @@sebastiankamph
I guess my 2go gpu won’t survive to this 🤣
Sadly not 😅
still dont understand why people get fedup with comfy, its way better for devs, more modular and easier to accomplish results...
Most people are just to lazy to learn, which is the case for every industry, not just AI. Comfy is a million times better if you know what you're doing and it's extremely light-weight compared to A1111.
@@CAPCOMhead And for whatever you're generating with AI, somebody could say you were too lazy to develop the artistic talent to make it yourself. But that's not entirely fair right? Fact of the matter is that learning anything is an opportunity cost, some of us have other interests, hobbies, software and tools to keep on top of. Time we spend deep diving into a single complex AI tool (that ends up being only a small part of our workflow) doesn't always make sense. Some of us prefer a simpler tool that gets out of the way, and for that are willing to give up some of the newest greatest features. Although I can see the appeal of Comfy for an AI enthusiast or somebody with more time to devote to it.
For my part I don't like spending a couple minutes stringing up modules whenever I want to maybe add a lora or a controlnet or something else until I've finally come up with an image that looks like something I want to put effort into refining. With A1111, that's only a few seconds and done.
I like it better for certain things and Auto/Vlad or Fooocus better for certain things. I use all of them daily lol.
you can build parts once, then save them as template :)@@Steamrick
Why would you remove my comment, this is not the true AnimateAnyone from Alibaba, the true one is not yet available
The one you are presenting is something trying to get close to it but it's definitely not it
I don't remove comments. If they get removed, you were using language RUclips doesn't support.
@@sebastiankamph My bad then sorry for the mistake
Don't blindly follow this video ~ read the comments below and the full text above by Sebastian first or you may be heading into a world of hurt.
So auto1111 is dead? No one does videos for it anymore. I have zero interest in learning comfy
It's not dead, certainly not for me. There are still YT'ers making videos on it but it's not advancing in the way that something like Comfy is, so there's not as much to talk about.
You have no interest in learning comfy? That's no one's problem but your own.
I also prefer a1111 but fact is comfy excels at certain things plus it's not really that hard as you can literally copy and paste a workflow in and you don't need to faff about much with all the noodles.
Like the others already mentioned, auto1111 don't get the new tech like Comfy as fast. So all the a1111 videos are on older stuff currently.
Sorry, but if you aren’t into learning then AI is def not the right space to be in.
Can you run comfy over network on your phone? My a1111 setup is too comfortable for the moment as i can prompt on my phone with my pc in another room.
good, but hands....
Is Automatic 1111 dead? None of the recent AI features have been added there... I hate ConfyUI...
Not dead, just that updates are less frequent these days. I would guess they test for stability more before pushing now. Or I hope :D
“With one picture you can make someone dance”. Yup, AI is causing stupidity.
Did you use AI to make this comment? No? Then what's your excuse for stupidity?
A111 can make like this video or just comyui only?
Garbage
Hmm... tried a lot but still getting errors w/these nodes.
"Cannot import D:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateAnyone-Evolved module for custom nodes:"
"Efficiency Nodes: Attempting to add 'AnimatedDiff Script' Node (ComfyUI-AnimateDiff-Evolved add-on)...Failed!"
"0.4 seconds (IMPORT FAILED): D:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateAnyone-Evolved"
Latest error, after doing a new install
File "D:\ComfyUI2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateAnyone-Evolved
odes.py", line 11, in
from diffusers.schedulers import (
ImportError: cannot import name 'LCMScheduler' from 'diffusers.schedulers' (D:\ComfyUI2\ComfyUI_windows_portable\python_embeded\Lib\site-packages\diffusers\schedulers\__init__.py)
Cannot import D:\ComfyUI2\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateAnyone-Evolved module for custom nodes: cannot import name 'LCMScheduler' from 'diffusers.schedulers' (D:\ComfyUI2\ComfyUI_windows_portable\python_embeded\Lib\site-packages\diffusers\schedulers\__init__.py)
fuck comfi, lets automatic 1111
The custóm workflow doensnt work because the 3D models cant be loaded! Wasted my money!! [ComfyUI-3D] Load UNet2D ConditionModel
[ComfyUI-3D] Load UNet3D ConditionModel
[ComfyUI-3D] Load Pose Guider
[ComfyUI-3D] Pose Guider Encode not available