ComfyUI AnimateDiff Prompt Travel: Runpod.io Cloud GPUs Tutorial
HTML-код
- Опубликовано: 1 окт 2024
- This video is a complete start to finish guide on getting ComfyUI setup with the addition of the ComfyUI-Manager and AnimateDiff with Prompt Travel on runpod.io
I am not sponsored by runpod.io at all. Here is the referral link associated with my account:
runpod.io?ref=liuedypr
The repos used are located here:
github.com/com...
github.com/ltd...
The workflow file can be downloaded from here:
drive.google.c...
The model (checkpoint) downloaded for this tutorial series is here:
civitai.com/mo...
The VAE used can be downloaded from:
huggingface.co...
The motion_modules can be found on here:
civitai.com/mo...
Socials:
x.com/c0nsumpt...
/ consumeem
/ c0nsumption_
Just a heads up. The ComfyUI used by Runpod seems a bit outdated and it (for me) didn't recognize the Manager installation.
Executing "git pull" inside the ComfyUI folder fixed this for me.
I literally just spent 2 hours trying to figure out what the problem was! It looked like a bug on the frontend with Comfy but ofc updating did the trick.
I mention it at some point in the vid! :) Thanks for dropping the comment. I will be updating this tutorial very very soon
this comment saved my ass
Another heads up! Not sure who needs to hear this (probably a lot of people) but here's what you need to do in runpod if insightface (via IPAdapter face models) isn't working:
In your virtual environment, run
apt-get update
apt-get install -y build-essential
pip install onnxruntime onnxruntime-gpu insightface
This comment will save asses for sure
@@AlphaBotCalls
waiting, it was working a moth ago how in this tutorial, but not now? sadness @@c0nsumption
The error message "bash: ls: command not found" indicates that the ls command, which is commonly used to list directory contents
Yo this is an amazing video... one small ( HUGE ) thing.... it's much better to purchase Network Storage and do everything you just did... THAT way you can move around to different GPUs on the fly and bring all that configuration, checkpoints, models, etc with you.... If you do not use Network Storage then you will have to go through your entire process in this video each time you want to purchase a new GPU...Just a heads up man... keep the videos coming. Network Storage costs you like ($0.07/GB/Month, $0.05/GB/Month for 1TB or more) when you are not using a GPU pod..
Awesome. Better to keep it simple though. Not many technical savvy people jumping in. Trying to get as many people possible using this new tech so I have to give bite sized chunks or else they get scared. Even having the video be 20 min is a huge deterrent lol.
I will keep this in mind though
Totally understand... it just an additional 2 steps in the beginning.. saves you loads of time later... @@c0nsumption
Can you use network storage with community cloud?
@@c0nsumption Great tutorial -- novice with Runpod, but would love to know how to save the workspace before terminating the pod. Using community GPUs.
a bit late but yes, I have been doing that since sunday@@koalanation
Help needed: I don’t see comfyui template on runpod. Looks like it’s gone or no longer available. What should I do, or as an alternative which template could I use instead of the comfyui template? Thanks in advance for any help!
Technically you would just need to git clone ComfyUI in and proceed with installation instructions. I’ll be updating this soon 🙏🏽
@@c0nsumption thank you!
thank you. your explanation is very neat and comprehensible. 1 Question i have is does installed nodes by Manger stays the same when I start over?(apart from ffmpeg)
Ok.The problem is some of my dependencies in Friz nodes disappear randomly after I shut down the GPU. I think that I fixed the problem after 3rd time building the image generator. the problem was likely that after going back to workstation after shutting down GPU for the day, I didn't activate venv again. It probably corrupted the files this way. You should have said that we had to do it every time we access our workstation.
thank you so much for this! excellent explanation and walk through, youre a legend!
I found you need quotes around the WGET parts
Great video thanks! Question, why in the video you mentioned to not open (click) on the port 3000?!
please help!
Now this error has started to appear at the apply ip adapter stage:
ERROR:root:Traceback (most recent call last):
File "/workspace/ComfyUI/execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/workspace/ComfyUI/execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/workspace/ComfyUI/execution.py", line 78, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/IPAdapterPlus.py", line 654, in apply_ipadapter
self.ipadapter = IPAdapter(
File "/workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/IPAdapterPlus.py", line 279, in __init__
self.image_proj_model.load_state_dict(ipadapter_model["image_proj"])
File "/workspace/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:
\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
how to fix it?
Thank you for the video. I can't see base model when I try to choose on the interface of comfyui, it is saying null. I have wget model from civitai but it is not showing up.
Thank a lot, great video, very useful info!
dude you're the shit! I Dont think ill be renting GPUs but I love all your videos!
Is there an update on this workflow? It isn't working for me.
First time i tried this , i made a half-ass-follow-thru .... and my ComfyUI-Manager did not work as it should.
Awesome tutorial
Thank you so much! Saw your premiere and instantly prepaid for runpod 😅This is pure gold. Is there a way to automatically test new prompts with different models? Now i manually form a queue, but I’d like to make a list of prompts and a list of models, basically XYZ for nodes
Give me a couple days and I’ll drop a video on this. Bit of a loaded question
Wow! Thanks so much for this detailed guide! Could you please share the animdiff worflow? the json file i mean, thanks in advance!
thanks bro, mind adding timestamps and chapters? easier to come back to the specific parts
Yeah, Il figure out how tomorrow. Super tired after Muay Thai 🧍🏽♂️
Wonderful tutorial, thank you!
so, we dont push button "terminate" . . . its right?
Hello, Thank you for well timed and detailed tutorials!
Unfortunately, at the last step of course I received an error stating my motion module is not compatible?
Error occurred when executing ADE_AnimateDiffLoaderWithContext:
('Motion model motionmodule15v2.ckpt is not compatible with SD1.5-based model.', MotionCompatibilityError('Expected biggest down_block to be 3, but was 2 - motionmodule15v2.ckpt is not a valid AnimateDiff-SD1.5 motion module!'))
File "/workspace/ComfyUI/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/workspace/ComfyUI/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/workspace/ComfyUI/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/nodes.py", line 112, in load_mm_and_inject_params
mm = load_motion_module(model_name, motion_lora, model=model, motion_model_settings=motion_model_settings)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/motion_module.py", line 230, in load_motion_module
raise ValueError(f"Motion model {model_name} is not compatible with SD1.5-based model.", e)
NVM I downloaded the latest beta for that motion module's SDXL version by mistake.
Many thanks for this! 🥰
I managed to get a pod to work last week by combining a few workflows but that was such a mess. This workflow is so much cleaner and doesn't run into all the low disk errors I got.
Awesome. Happy to help 👏🏽
Is it possible to initiate a Runpod VM with limited GPU resources, and once the setup is done / user interface (UI) is configured, can you migrate it to a pod with higher GPU capabilities?
I will be doing a deep dive very soon brother. I am launching my channel again as of tomorrow. Runpod will be a huge topic. One experiment we will be doing is hooking up Runpod to an external storage system to see if we can then accomplish what you’re saying:
- set up instance
- connect to storage medium
- run workflow.
Forgive my tardiness, I have been in deep research with a couple businesses and am bringing Cloud Computing, AI, Spatial Computing, and much much more as the primary focus of my channel.
Tomorrow will be an announcement and updates video 🙏🏽
Greetings. thanks for your work. When I activate load checkpoint on comfyui, it shows null, base models are not shown, what can I do?
Sorry for delayed response. I will be refilming this to update the tutorial.
You must ensure that your checkpoint fully downloaded when doing wget and that it was named properly 🙏🏽
Super. good channel- see ya-
Thanks sir 🙏🏽
why am I getting a "No space left on device" error when installing pytorch?
Your pod doesn’t have enough storage space. You need to adjust the amount of storage they are giving you. I’m gonna need to update this.
Enjoy all your vids, clearly explains what to do and helps get things up and running.
Thanks for the fantastic video
I followed everything perfectly up until 18:11 and got an error when queuing the prompt:
Error occurred when executing ADE_AnimateDiffLoaderWithContext:
('Motion model motionmodule15v2.ckpt is not compatible with SD1.5-based model.', MotionCompatibilityError('Expected biggest down_block to be 3, but was 2 - motionmodule15v2.ckpt is not a valid AnimateDiff-SD1.5 motion module!'))
Is this because the motion module is missing an extension?
Thank you for making this tutorial. The way you explain is super clear and great to follow 😊
Quick question related to Runpod. I have been only using this method to connect with my runpod. THANK YOU!
I wanted to share the static url they provide with a colleague but the Runpod will not keep or install VHS_VideoCombine... Any thoughts?
Thank you for this information!
How much VRam would I need to run Steerable motion?
I will be dropping a video on this very soon actually, stay tuned. Haven’t tested on new updates!
Good tutorial, i was trying to get facefusion work with the same steps but im unable to connect with runpod's ip:port. I also added the facefusion's default port.
Hmmm, interesting. I’m currently tied up but I’ve been researching Runpod extensively. Videos and live streams coming :) Been delayed cause of many interviews and connections, forgive me 🙏🏽
No worries! i found docker template for facefusion, so it's working for now at least.@@c0nsumption
Error occurred when executing ADE_AnimateDiffLoaderWithContext:
Motion module 'motionmodule15v2.ckpt' is intended for SDXL models, but the provided model is type SD1.5.
File "/workspace/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/workspace/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/workspace/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/nodes_gen1.py", line 138, in load_mm_and_inject_params
motion_model = load_motion_module_gen1(model_name, model, motion_lora=motion_lora, motion_model_settings=motion_model_settings)
File "/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/model_injection.py", line 424, in load_motion_module_gen1
raise MotionCompatibilityError(f"Motion module '{mm_info.mm_name}' is intended for {mm_info.sd_type} models, " \
nvm ... downloaded bad version, fixed and working 20-04
wow, you are amazing, i fallowed the instructions and it is working; i aprecite a lot your energy and time you spend sharing al this infos; for people without programming background all this things looks like Chinese. cheers
Thanks dude 🙏🏽
Going to be making some more Runpod tutorials to show you guys the level of versatility you actually have there. Considering doing live streams 🤔
@@c0nsumption It would be grate to have live streams; i know you mentioned at a point in your video that you are thinking of making a channel on discord. If you have time, i encourage you to do it cause you are starting to build a community.
Would setting up on AWS (maybe EC2 instance with G5) follow similar workflow?
I was looking for the god in the wrong place, he was here all along.
😂 you’re too kind
I dont know why, but the comfy ui manager button doesnt appear in the UI.
‘git pull’ in the ComfyUI root folder. Needs to update so that it can be compatible with the manager
@@c0nsumption Thanks a lot man it helped.
Hello!
Your ComfyUI tutorials are great and I always learn by watching your videos.
I have one request, if possible, could you please tell me how to use FaceDetailer (Stable Diffusion=Adetailer) with animatediff in ComfyUI?
I'm still having difficulty finding a good workflow for using FaceDetailer with the animatediff of ComfyUI.
Could you posibly make a tutoriol on how to run it serverless?
THanks for the video. can you do it for kaggle ?
thanks alot eh well worth the sub
Brilliant, have been thinking about this
Amazing video! Would love more simple runpod tutorials like this. Sometimes my ComfyUI seems to disconnect from the Jupyter terminal and it will show images generating in Jupyter, but nothing appearing in ComfyUI. Did I do any of the steps wrong you think?
🤔 Could just be server lag. Or some kind of bug with the server itself.
i cannot install. this is a scam i bet how can i get money back.... it just stops installing for no reason and i cant do anything
I already said you need to increase your drive size.
@@c0nsumption when i terminate the pod i will lose all these hours of downloading stuff? wow.
i cannot install new checkpoints, i install them and they never show up on comfy ui........
my bro, that was awesome....thanks man 🙌
Amazing tutotial !!!! Keep them coming i will be checking you channel a lot from now on. Can you share the workflow file ? I tried to download it but the link goes to a diagram png image
You can use the png file :)
much appreciated!!
Hi
i have that error in my local Comyui "[AnimateDiffEvo] - WARNING - ffmpeg could not be found. Outputs that require it have been disabled"
how i can fix it please :)
Watch from 16:30 forward. Make sure to finish the video, there are important details on it
yes i tryed what did you do in the video but this error happend
apt : The term 'apt' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ apt install ffmpeg
+ ~~~
+ CategoryInfo : ObjectNotFound: (apt:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
@@c0nsumption
I'm talking about confyui in my PC not in cloud @@c0nsumption
Should i download or insatll something cus i'm a noob in this @@c0nsumption
@@SheRoMan chatgpt is saying that’s a powershell error and not possible on a Linux server
Just wonderful!
Just wonderful!
Hey nice artwork! Just want to ask that are you interested in our new ai generation product? Maybe I can send you beta version to you~
Contact me on X:
@c0nsumption_
Amazing! Thank you so much!
Does it save everything you did like new models and ffmpeg setup when you stop the pod? So you don't need to do all that everyday. Do they charge for keeping the data/pop while it's not running?
With questions like this:
Please watch video first. This is what the video is about. This is touched on multiple times in different sections.
16:30 till just about the end
as of sept - to maintain the environment when starting and stopping the pod you need to use network volumes...but right now the template is not updated so still deletes the venv folder...i contacted runpod support - they said to use the better comfy ui light template.
couldnt install requirements.txt, gave me this error
" ━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━ 516.0/731.7 MB 183.0 MB/s eta 0:00:02ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device"
what a scam this website????
You need to make your drive larger, not enough storage!
Delete your pod! Make sure to increase storage config to something more useable 🙏🏽
Thank you so much for this. As somebody who is an absolute beginner in all this, your video was extremely helpful. I'm wondering, is it sometimes worth it to never completely terminate your pod? I would like to use runpod extensively, with various types of workflows (various nodes,models, etc). But I do not want to have to set everything up everytime. Because it's relatively cheap, I might just consider leaving the pod turned off, but not completely terminate it. Or do you have another suggestion? Thanks again! subbed!
There are way to actually offload that stuff to the cloud and connect your cloud storage to Runpod instead. So technically you won’t always have to set up.
I’ll be covering things like this extensively. Been researching Runpod for a couple weeks now
@@c0nsumption yeah I second this. Would love a video that delves into a runpod workflow that start everything up again after terminating, without too much work/scripting.
@@c0nsumption would love a link to this here!
Great tutorial! Just a note: the link to the workflow file doesn't point to a json but to a png image of the workflow
Should work the similar in the mean time. Thanks for giving me the heads up, I’ll fix it in an hour! Just got home 😳
@@c0nsumption yes the workflow would help me out a lot! Thank you so much for the tutorial btw it helped me quite a lot!
oh nevermind you can just drag and drop the png file haha
yeeyyy