fantastic work!, the only problem is that I found that after running this workflow, the background is always pure color, even if I have added some background info in prompt, it does not work, could you share some fix method?
Thanks for the feedback! First I think you can try to bypass the "AnimateDiff Loader" and set "Input_Img_Cap" to 1, and run some single picture to check if the background is generated as you wish, change to different models if not. Or you can add the depth controlnet with a very low strength, 0.2 for example. If you just want to restyle the source video, you can decrease the denoise value in the first Ksampler, like 0.6 - 0.8.
Thank you for the video.but i have the problem :Error occurred when executing VHS_LoadImagesPath: directory is not valid: E:\Ai_video\ComfyUI_windows_portable\ComfyUI\output\ADiff\OpenPose, I followed the instructions and created the folders.thanks for help.
Strange, have you updated VHS to the latest? Please make sure it's not "Update error" for VHS in the ComfyUI CMD window. If it still doesn't work, you can try "Load Image Batch from Dir" in ComfyUI-Inspire-Pack, that's a substitute I tried before and it also works @@MacacoNinja
thansk so much for ur great tutorials.is there any render time limit in comfyui? i wanna use a 32seconds video.its 30 frame (1000 png) for video2video but i got this error on my 3090ti: numpy.core._exceptions._ArrayMemoryError: Unable to allocate 6.43 GiB for an array with shape (976, 1024, 576, 3) and data type float32
It seems 1000 might be a little bit more, I didn't have a chance to try that yet, but you can try a lower fps, like 30fps -> 10fps, and do frame interpolation (ComfyUI-Frame-Interpolation VFI node) afterwards, thus you only need to generate 320 images a time. I have a VFI node example in my RAVE video
@@JanRTstudio numpy.core._exceptions._ArrayMemoryError: Unable to allocate 6.43 GiB for an array with shape (976, 1024, 576, 3) and data type float32 i got this error
Thanks, the models should be put in ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\models, you need to install ComfyUI_IPAdapter_plus from the Comfyui manager first. I updated both links in the descriptions.
I don't have a specific one to share right now, RUclips should have plenty of them. If you are familiar w/ sdwebui A1111 then just have some practices to build the workflow, otherwise I suggest to try sdwebui first, you will get familiar with the parameters and then much easier in comfyui@@keepitshort4208
To Run is need: ShowText|pysssss (In group node 'workflow/Dimensions') workflow/DimensionsRemove from workflow CR Prompt Text Image Save Text Multiline CR Split String Display Int (rgthree) ColorMatch CR Seed ¿where is there?
Thanks for this vid and great resource. For whatever reason, it is not loading an image into the load image portion of the ipadapter. Everything else prior to that step seems to be working fine. Do you have any ideas what might cause that? Comfy is not throwing any errors.
Thanks for the feedback! Can you try to delete and add again the load image node, also click "Refresh" on the side menu? Also is the image in the comfyui Input folder already? loaded image will be copied to that folder. That's all I can think of now.
That's pretty cool! 24G is a lot, it's mainly GPU ram to render, if shifted to CPU ram the speed will slow down dramatically, so 32G RAM is good enough you don't want to use it. With 24G you can try latent upscale to get high resolution animation. I just have 12G VRAM and can do 512x768 100frames+ one time. You are good to go!
For some reason, even though I've successfully downloaded Comfyui Impact Pack, Comfyui will still say it's missing. So the node above SAMLoader in the Face Detailer section is red. Have you run into this issue?
That's strange, did you use comfyUI manager to install Impact? Just check the CMD window, during loading it will show "Import Failed" for Impact Pack, and before that, it actually gives you error and the cause of failure.
Sure, did you check the error message from the CMD window? Seems can't find the image. Probably you need to click the Refresh on the side menu, and select/drag the image again to the node.
Is there any way to make mouth move as in original video clip? Any advice or setting i could tinker with? Or is this just good with movement and dances.
It depends on your PC GPU & memory. It took about 2min and a half for 62 frames in this example. Too many frames might run out your VRAM and shared memory but yes, you can always render it clip by clip.
Error occurred when executing VHS_LoadImagesPath: directory is not valid: D:/Program Files/ComfyUI_windows_portable/ComfyUI/output/ADiff/JanRT_P05/ File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 155, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 85, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 78, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite\videohelpersuite\load_images_nodes.py", line 143, in load_images raise Exception("directory is not valid: " + directory) I've tried many times but I really don't know how to modify
You can change the first CR prompt text to your folder path "D:/ComfyUI_windows_portable/ComfyUI/output/ADiff/", and rerun from the beginning. Loading images from folder other than the "output" folder inside ComfyUI usually causes error, I think that's the reason.
My brain always shuts down when I see ComfyUI. I started a week ago on a1111 so yeah, very early days!
lol yeah get familiar with a1111, you will find it's just those options separated as nodes in comfyui
Thank you for the video. I really like the way you parsed the strings to control the locations of content. Very cool idea.
That’s great to hear! Appreciate your feedback!
@@JanRTstudioThanks also for sharing the workflows! Makes learning so much easier.
You are welcome! @@JKG-777
I was having problems setting it up but then I restarted the process from start and it worked, thanks a lot for the video!
sure, you are very welcome!
Succinct and clear.
Thanks!
Thank you Thank you Thank you.
😀
fantastic work!, the only problem is that I found that after running this workflow, the background is always pure color, even if I have added some background info in prompt, it does not work, could you share some fix method?
Thanks for the feedback! First I think you can try to bypass the "AnimateDiff Loader" and set "Input_Img_Cap" to 1, and run some single picture to check if the background is generated as you wish, change to different models if not. Or you can add the depth controlnet with a very low strength, 0.2 for example. If you just want to restyle the source video, you can decrease the denoise value in the first Ksampler, like 0.6 - 0.8.
Error occurred when executing KSampler:
'NoneType' object has no attribute 'shape'
Might miss the inputs or something mismatches, anything changed? do you have more info from the Traceback?
Thank you for sharing. Great workflow. Setting up the workflow is really hard.
Thanks for your feedback! Glad you like it!
Thank you for the video.but i have the problem :Error occurred when executing VHS_LoadImagesPath:
directory is not valid: E:\Ai_video\ComfyUI_windows_portable\ComfyUI\output\ADiff\OpenPose, I followed the instructions and created the folders.thanks for help.
Can you try changing all "\" in the path to "/" in the first prompt text box? thanks
@@JanRTstudio I have the same problem, I already changed from \ to / but I have the same error, for some reason it doesn't seem to find the directory.
Strange, have you updated VHS to the latest? Please make sure it's not "Update error" for VHS in the ComfyUI CMD window. If it still doesn't work, you can try "Load Image Batch from Dir" in ComfyUI-Inspire-Pack, that's a substitute I tried before and it also works @@MacacoNinja
Thank you it works great, just something wrong with colors. They are too bright and the image is overexposed. Have you an idea what's wrong?
Sure! It might be the color match node, you can try different reference picture for the color match
When loading the graph, the following node types were not found:
IPAdapterApply
I have fully installed the nodes and still get this error
thansk so much for ur great tutorials.is there any render time limit in comfyui? i wanna use a 32seconds video.its 30 frame (1000 png) for video2video but i got this error on my 3090ti: numpy.core._exceptions._ArrayMemoryError: Unable to allocate 6.43 GiB for an array with shape (976, 1024, 576, 3) and data type float32
It seems 1000 might be a little bit more, I didn't have a chance to try that yet, but you can try a lower fps, like 30fps -> 10fps, and do frame interpolation (ComfyUI-Frame-Interpolation VFI node) afterwards, thus you only need to generate 320 images a time. I have a VFI node example in my RAVE video
@@JanRTstudio numpy.core._exceptions._ArrayMemoryError: Unable to allocate 6.43 GiB for an array with shape (976, 1024, 576, 3) and data type float32
i got this error
Great video
I have a question, Can you tell me the location for the *IPAdapter model* in comfy ui file
Thanks, the models should be put in ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\models, you need to install ComfyUI_IPAdapter_plus from the Comfyui manager first. I updated both links in the descriptions.
@@JanRTstudio appreciate it 🙏
Do you have a video on learning comfy ui, as in a step by step or you recommend someone that does ?
I don't have a specific one to share right now, RUclips should have plenty of them. If you are familiar w/ sdwebui A1111 then just have some practices to build the workflow, otherwise I suggest to try sdwebui first, you will get familiar with the parameters and then much easier in comfyui@@keepitshort4208
@@JanRTstudio appreciate the reply 👍
To Run is need:
ShowText|pysssss (In group node 'workflow/Dimensions')
workflow/DimensionsRemove from workflow
CR Prompt Text
Image Save
Text Multiline
CR Split String
Display Int (rgthree)
ColorMatch
CR Seed
¿where is there?
You can simply install these custom nodes in Comfyui manager (github.com/ltdrdata/ComfyUI-Manager), click "Install Missing Custom Nodes"
Thanks for this vid and great resource. For whatever reason, it is not loading an image into the load image portion of the ipadapter. Everything else prior to that step seems to be working fine. Do you have any ideas what might cause that? Comfy is not throwing any errors.
Thanks for the feedback! Can you try to delete and add again the load image node, also click "Refresh" on the side menu? Also is the image in the comfyui Input folder already? loaded image will be copied to that folder. That's all I can think of now.
i upgraded to 3090ti 24gig.how much cpu ram i need to do video to video SD? I have 32gig
That's pretty cool! 24G is a lot, it's mainly GPU ram to render, if shifted to CPU ram the speed will slow down dramatically, so 32G RAM is good enough you don't want to use it. With 24G you can try latent upscale to get high resolution animation. I just have 12G VRAM and can do 512x768 100frames+ one time. You are good to go!
For some reason, even though I've successfully downloaded Comfyui Impact Pack, Comfyui will still say it's missing. So the node above SAMLoader in the Face Detailer section is red. Have you run into this issue?
That's strange, did you use comfyUI manager to install Impact? Just check the CMD window, during loading it will show "Import Failed" for Impact Pack, and before that, it actually gives you error and the cause of failure.
Thank you for the workflow, but when i run the workflow, the load images node will get error, turn red, can you tell me how to solve it
Sure, did you check the error message from the CMD window? Seems can't find the image. Probably you need to click the Refresh on the side menu, and select/drag the image again to the node.
Is there any way to make mouth move as in original video clip? Any advice or setting i could tinker with? Or is this just good with movement and dances.
Yes, that's also something I am thinking about, batch prompt might help, will see if I can get it and share. Thanks for your feedback!
how long does it take for render, and can we render a full dance video?
It depends on your PC GPU & memory. It took about 2min and a half for 62 frames in this example. Too many frames might run out your VRAM and shared memory but yes, you can always render it clip by clip.
thanks for your Info @@JanRTstudio
This is too hard to follow. You should build it from scratch with every step shown.
OK, I will consider it in the next video, thanks for the feedback
Error occurred when executing VHS_LoadImagesPath:
directory is not valid: D:/Program Files/ComfyUI_windows_portable/ComfyUI/output/ADiff/JanRT_P05/
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 78, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite\videohelpersuite\load_images_nodes.py", line 143, in load_images
raise Exception("directory is not valid: " + directory)
I've tried many times but I really don't know how to modify
You can change the first CR prompt text to your folder path "D:/ComfyUI_windows_portable/ComfyUI/output/ADiff/", and rerun from the beginning. Loading images from folder other than the "output" folder inside ComfyUI usually causes error, I think that's the reason.
Thank you for the video. Please help me to download the clip vision model ViT-H.safetensors.
Sure, check here: huggingface.co/h94/IP-Adapter, the first Image Encoder model
@@JanRTstudio hi having same problem cant find "ViT-H.safetensors". cant find the file in above link as well
I renamed it, just updated the direct links of model (same h94 huggingface) in the description, you can download it@@OtakuOrakle