Tonight playing with a workflow I found I could get someone to (kind of) walk by getting images in the right order, this kind of baffled me, I sit down and put TV on and see this. Thankyou so much for showing me what my workflow is telling me is possible. Many thanks for all your contributions.
I keep getting this error: "Prompt outputs failed validation - IPAdapterBatch: - Exception when validating inner node: tuple index out of range" EDIT: I did an update all and now this error is gone, but got a new one. "Error occurred when executing IPAdapterBatch: cannot access local variable 'face_image' where it is not associated with a value" If I bypass the 2nd IPAdapter node, it works? So something it doesn't like w/that node. EDIT: The problem was the IPAdapter weights was set to "full batch" instead of alternate. So was wasn't getting any images for the 2nd IPAdapter.
Seems like a great base to use when upscaling video. Upscale the key frames but also utilize the original animation for controlling pose or whatever. Very cool technique
This is great tutorial. As a newbie to ComfyUI I found there were a lot of additional things I needed to download that weren't mentioned such as Clip Vision 😉
@@latentvision Thank you, I should have watched the video before asking the question :) Your videos and time developing these nodes is of huge benefit to the open AI community, thank you!
6:54 hey Matteo I extracted frames from video and placed the frames into a folder. Instead of using 'Load Image' node one by one, Is there any node automatically load up images from a folder in order? like files names are in order so it can load up images automatically in order. Thank you always.
Great! Thanks also for sharing your files. Now i am waiting for ipadapter wich can handle higher resolution and also waiting for more context length with animatediff.
Hey Matteo, sorry another annoying question from me. Your workflow works a charm and I'm having great results with the typography workflow. I've been trying to create a moment at the beginning before the first word comes in. I can do this by adding a black image in the Images Batch Multiple node before the first word. But the result is that there is no 'die off' after the second word. I've tried many things; adding 2 black frames at the end, repeating the second prompt 3 times in the Prompt Schedule From Weights Strategy node, adding more frames in the IPAdapter Weights node, but nothing seems to work. Any thoughts would be helpful. I know you're not getting paid for this so I appreciate any help at all
hard to say without seeing your workflow. but generally speaking you need to add a "fire" frame at the beginning (so animation starts with 2 fire images basically) and then a black frame for the control net
I love the workflow! is there any chance to get less movement in your second example? LIke can I tell the AnimateDiff Node to decrease the movements from frame to frame?
This is brilliant--thank you for sharing! Is it possible to apply a style lora into the workflow? The IP adapter gets the look pretty close, but if a custom style lora could be applied in conjunction with the IP adapter that would push things to a whole new level.
What you think to add an image interrogator from the last images batch multiple and connect it to the Prompt Schedule? It will require string format but I guess it could work...
@@latentvision You're right. It turns out that I was let down by the habit of repeating what I saw from your videos without using ready-made workflows :).
Hello Matteo, Thank you for the great tool and tutorials! I haven a question. I am unable to use this technique mantaining the characteristics of the image I am using. For some reason the result comes different from the input I have created. What is the parameter thet controls how much of the input image is used? Can I force it to just follow it? Cheers
Cool! I need a lot of animation frames, so image cherry picking and manual keyframing just doesn't cut it, but this method works great for shorter and detailed animations. Suggestion: color code the nodes so it would be easier to follow. With all grey nodes, it is hard to follow, especially on mobile phone. I hope we will see more animation stuff soon;)
oh man Matteo Thank you so much this is what I have been looking for! possibly I could apply batches of mask to make an animation? like I get a sequence of water movement and get masks of the sequence. connect the masks to attention mask to create other objects moving mimicking water movement.
Hello Sir. Can you Please help me out? ipAdapter faceid suddenly got extremely slow and I have no idea Now to fix it. It did not use to be that slow. Do you have any idea what I could do?
Hey Matteo, thanks for the amazing job you're doing. Following this workflow i get an error: "only integer tensors of a single element can be converted to an index". This is happening when i turn the IPAdapter Batch nodes "weight" widgets to inputs and connect them to the IPAdapter Weights node output. Somehow if i turn back those weights inputs into widgets, the Sampler is able to process them, but ofc i don't get the desired result. Do you know what this might be related to?
very informatic tutorial, when i running fire water workflow i am getting error from prompt sheduler which is missing 4 req positional arguments: pw_a, pw_b, pw_c and pw_d. pls suggest me what is the solution. thanks
Not sure if this was mentioned, but for the life of me I couldn't find the Images Batch Multiple node. Took a bit of searching (Manager was quite unhelpful, here) until I found it was part of the ComfyUI Essentials pack. Hope this helps someone.
Hey Matteo, I don't seem to find the "lcm-lora-sd15.safetensors" file anywhere online. I've followed your links in description but they bring me to .ckpt files, so I'm a bit confused here. Can you please help? Thanks a lot for your time.
I got a question about the IPAdapter Weights node. If you want to " hold" one of the input images for a while instead of constantly evolving, how would one approach this. You can increase the number of Frames used but it's still moving forward to the next input image, could you somehow freeze this for a few frames? Or am I asking to much now haha.
This tutorial is really great! Very practical!(sponsored!) But I have a small question: if I don't want the original image to change, which parameters do I need to adjust? I tried ControlNet, but it doesn't seem to work.
with animatediff the original image will always change to a certain degree. You can use video2video or controlnets, but it's not like SVD for example that it starts from a given frame and reiterate on that
If you hook it up to a Display Any node, you'll see what the outputs are. It looks like it's a list of parameters specific to Matteo's nodes in order to generate the appropriate keyframes. Essentially it's a parametric way of calculating the keyframes, that way you can add or remove images and it will automatically adjust the keyframes accordingly. This replaces the need to use something like Batch Prompt Schedule or Batch Value Schedule nodes to manually enter in keyframe values.
Hey Master Matteo! Trying here on a Mac Silicon.. In the end of the script, I see this error: "RuntimeError: MPS: Unsupported Border padding mode" Probably a Mac error? :(
Hmm, when trying to use your workflow I'm getting this error When loading the graph, the following node types were not found: IPAdapterBatch IPAdapterUnifiedLoader IPAdapterWeights IPAdapterNoise Nodes that have failed to load will show as red on the graph. I've updated ComfyUI_IPAdapter_plus , deleted and recloned, deleted and redownloaded through manager, and I continue to get the same error each time. no module named "node helpers" is why it fails to import
Is your ComfyUI up to date? That sometimes messes things up for me. Can try a git pull when inside the ComfyUI folder and after that try to update IPA again.
Странно, я полностью повторил это видео и у меня совершенно другой результат. Все модели теже. Но на выходе получается уродливое видео :( Этот метод больше не работает?
Someone should be paying this man.
LOL, I agree! :D
Can't thank you enough for your contributions to the field. You are truly a genius!
one "thank" is enough :)
This is history in the making.
damn, I'm so old already?!
Although, you still will see flicker and issues at higher detail resolutions. These are very simple examples
@@RetzyWilliamsits still really great ground building for the potential that others can do with this tool
Thank you for all your hard work, Brother! Your contributions to this community have helped to elevate my content so much. I can't thank you enough.
Amazing work Matteo as always.
Proud to share italian roots with some talented guys like you.
I've never clicked so fast on a RUclips thumbnail !
🤣
Tonight playing with a workflow I found I could get someone to (kind of) walk by getting images in the right order, this kind of baffled me, I sit down and put TV on and see this. Thankyou so much for showing me what my workflow is telling me is possible. Many thanks for all your contributions.
An absolutely awesome masterclass from Maestro Latente!!... so many great tips that I cannot thank you enough!!
I keep getting this error: "Prompt outputs failed validation - IPAdapterBatch:
- Exception when validating inner node: tuple index out of range"
EDIT: I did an update all and now this error is gone, but got a new one.
"Error occurred when executing IPAdapterBatch:
cannot access local variable 'face_image' where it is not associated with a value"
If I bypass the 2nd IPAdapter node, it works? So something it doesn't like w/that node.
EDIT:
The problem was the IPAdapter weights was set to "full batch" instead of alternate. So was wasn't getting any images for the 2nd IPAdapter.
same
same problem with "face_image" error. Thanks for the solutions in edit.
Thanks for the solution!
Legend thank you I had this same issue, thanks for posting the solution!
This looks totally mind blowing! Thanks for sharing! Would love to watch a breakdown that is suited more for beginners, especially for the later part.
Thanks Mateo for this great topic that I am not ready (yet!) at all
Seems like a great base to use when upscaling video. Upscale the key frames but also utilize the original animation for controlling pose or whatever. Very cool technique
This is great tutorial. As a newbie to ComfyUI I found there were a lot of additional things I needed to download that weren't mentioned such as Clip Vision 😉
Great stuff. Thank you very much for your knowledge. Have a good day!
I love you SO SO MUCH! Been waiting for this tutorial since I saw your post last week hahaha Thank you thank you
This is wonderful. Thank you always. This only works for SD1.5 models, correct?
there are a couple of SDXL models for AnimateDiff, but they don't work very well
When Matteo speaks, I listen👌👌
Kindly make about clothes and garments on modern
While I'm overwhelming about all the node and how the man know to use them
Mateo: this is only the tip of the iceberg 😳
Hey Matteo, thanks so much for this. Is thee a workflow for creating such consistent character images like you did with the blond girl?
as I said in the video it's mostly prompting, but if you add an IPAdapter of the first generation the subsequent will be very close to it
@@latentvision Thank you, I should have watched the video before asking the question :) Your videos and time developing these nodes is of huge benefit to the open AI community, thank you!
6:54 hey Matteo I extracted frames from video and placed the frames into a folder. Instead of using 'Load Image' node one by one, Is there any node automatically load up images from a folder in order? like files names are in order so it can load up images automatically in order. Thank you always.
check the node "load images path"
This is super, Matteo. Why are you so good at this?
Thanks!
The url in notes for the GIF controlnet model does not lead to that model unless these other motion models are the same thing by a different name.
just rename it
Thank you, I learn new stuff from here, all the love for you
Thank you for the in-depth video! But where do I get the ControlGIF model for the ControlNet node?
Great! Thanks also for sharing your files. Now i am waiting for ipadapter wich can handle higher resolution and also waiting for more context length with animatediff.
also i can not find controlGIF.ckpt file
any luck with it?
were you able to find the .ckpt file?
Nicee, thanks, you think vid2vid is coming soon?
My God, ... Mateo, my master, eternal respect to you, I am shocked by your knowledge.
I just hope my 1080Ti can handle this xD
Thanks one again!
Hey Matteo, sorry another annoying question from me. Your workflow works a charm and I'm having great results with the typography workflow. I've been trying to create a moment at the beginning before the first word comes in. I can do this by adding a black image in the Images Batch Multiple node before the first word. But the result is that there is no 'die off' after the second word. I've tried many things; adding 2 black frames at the end, repeating the second prompt 3 times in the Prompt Schedule From Weights Strategy node, adding more frames in the IPAdapter Weights node, but nothing seems to work. Any thoughts would be helpful. I know you're not getting paid for this so I appreciate any help at all
hard to say without seeing your workflow. but generally speaking you need to add a "fire" frame at the beginning (so animation starts with 2 fire images basically) and then a black frame for the control net
I love the workflow! is there any chance to get less movement in your second example? LIke can I tell the AnimateDiff Node to decrease the movements from frame to frame?
you can run it slower by increasing the number of frames
This is brilliant--thank you for sharing!
Is it possible to apply a style lora into the workflow? The IP adapter gets the look pretty close, but if a custom style lora could be applied in conjunction with the IP adapter that would push things to a whole new level.
couldn't these setups be packaged into the program so we just change the variables instead of going to such a steep learning curve?
they could, yes
OMG, This is a great job , thank you so much
What you think to add an image interrogator from the last images batch multiple and connect it to the Prompt Schedule? It will require string format but I guess it could work...
What is up with the Shutterstock watermark in the final image?
controlGIF is "motion_checkpoint_less_motion" or "motion_checkpoint_more_motion" from "crishhh/animatediff_controlnet" ?
normal motion :D
@@latentvision Most likely it's controlnet_checkpoint.ckpt from "crishhh/animatediff_controlnet"
I believe I put the link inside the workflow in a note node
@@latentvision You're right. It turns out that I was let down by the habit of repeating what I saw from your videos without using ready-made workflows :).
@@latentvision I don't understand your conversation. Which model is controlGIF?
Hello Matteo, Thank you for the great tool and tutorials! I haven a question. I am unable to use this technique mantaining the characteristics of the image I am using. For some reason the result comes different from the input I have created. What is the parameter thet controls how much of the input image is used? Can I force it to just follow it? Cheers
Cool! I need a lot of animation frames, so image cherry picking and manual keyframing just doesn't cut it, but this method works great for shorter and detailed animations.
Suggestion: color code the nodes so it would be easier to follow. With all grey nodes, it is hard to follow, especially on mobile phone.
I hope we will see more animation stuff soon;)
Does this work for images without people? For example for making a video of clouds flowing
sure, it works with anything
oh man Matteo Thank you so much this is what I have been looking for! possibly I could apply batches of mask to make an animation? like I get a sequence of water movement and get masks of the sequence. connect the masks to attention mask to create other objects moving mimicking water movement.
yeah that would work too
thank you for this helpful tutorial
i dont have sgm_uniform as a scheduler. can someone point out how/what to get this?
Hello Sir. Can you Please help me out? ipAdapter faceid suddenly got extremely slow and I have no idea Now to fix it. It did not use to be that slow. Do you have any idea what I could do?
please join my discord or post an issue on github, it's hard to escalate on an youtube comment
@@latentvision i understad that. You are right. I will join the dicord and post it as an issue. Thank you for your Work.
Hey Matteo, thanks for the amazing job you're doing.
Following this workflow i get an error: "only integer tensors of a single element can be converted to an index".
This is happening when i turn the IPAdapter Batch nodes "weight" widgets to inputs and connect them to the IPAdapter Weights node output.
Somehow if i turn back those weights inputs into widgets, the Sampler is able to process them, but ofc i don't get the desired result.
Do you know what this might be related to?
please post an issue on the official repository adding workflow and complete error message
very informatic tutorial, when i running fire water workflow i am getting error from prompt sheduler which is missing 4 req positional arguments: pw_a, pw_b, pw_c and pw_d. pls suggest me what is the solution. thanks
Not sure if this was mentioned, but for the life of me I couldn't find the Images Batch Multiple node. Took a bit of searching (Manager was quite unhelpful, here) until I found it was part of the ComfyUI Essentials pack. Hope this helps someone.
I'm getting a 'TypeError: can't multiply sequence by non-int of type float' when I try your workflow?
Hey Matteo, I don't seem to find the "lcm-lora-sd15.safetensors" file anywhere online. I've followed your links in description but they bring me to .ckpt files, so I'm a bit confused here. Can you please help? Thanks a lot for your time.
search LCM LORA on huggingface
Thanks as always ... I have a question .. Can we make it loop video?
there's a way to make kinda looping videos in animatediff, check the main repository
Getting this error, any idea why? Required input is missing: encode_batch_size
you probably just need to refresh the page
There needs to be a frame that is perfectly from behind. Otherwise you'll get that crazy Popeye-jaw.
Someone knows the node for his "images batch multiples" and "ipadapter weights" ? thk you
comfyui essentials and ipadapter of course
@@latentvision thk you for the answer and your work.
Another question... why you don't use the node "everywhere" ? Did you encoutered trouble with it ?
@@seminole3001 it makes the workflow very difficult to follow especially when teaching. In a node system like comfy it's considered an "anti-pattern"
@@latentvision last question, the model animateGIF ? Did you rename it ? I don't find a link to download it...
I got a question about the IPAdapter Weights node. If you want to " hold" one of the input images for a while instead of constantly evolving, how would one approach this. You can increase the number of Frames used but it's still moving forward to the next input image, could you somehow freeze this for a few frames? Or am I asking to much now haha.
the easiest is to repeat the frame twice
This tutorial is really great! Very practical!(sponsored!) But I have a small question: if I don't want the original image to change, which parameters do I need to adjust? I tried ControlNet, but it doesn't seem to work.
with animatediff the original image will always change to a certain degree. You can use video2video or controlnets, but it's not like SVD for example that it starts from a given frame and reiterate on that
What is the software called when you were refining the images?
it's an open source software called GIMP
I'm not sure but I think Mateo mentioned gimp in one of his earlier video's
Good open source tools are also photopea and Krita 😉
Can someone explain how the weights strategy parameter works?
If you hook it up to a Display Any node, you'll see what the outputs are. It looks like it's a list of parameters specific to Matteo's nodes in order to generate the appropriate keyframes. Essentially it's a parametric way of calculating the keyframes, that way you can add or remove images and it will automatically adjust the keyframes accordingly. This replaces the need to use something like Batch Prompt Schedule or Batch Value Schedule nodes to manually enter in keyframe values.
Why sgm_uniform? Karras worse?
If I remember correctly it is recommended with LCM sampler
How do we find that Discord server you mentioned at the beginning?
try this discord.gg/WdpGf2tx
You mention a Discord channel for animation (Bannadoku or somthing - its hard to hear). Can you provide a link or the correct name?
banodoco, see you there :D
@@elowine When searching Discord communities for banadoco I get zero hits. Do you need an invite link to find it?
Hey Master Matteo! Trying here on a Mac Silicon..
In the end of the script, I see this error:
"RuntimeError: MPS: Unsupported Border padding mode"
Probably a Mac error? :(
please report the error on github, posting the full backtrace. thanks
in the first pass how did u get consistent characters
Hmm, when trying to use your workflow I'm getting this error
When loading the graph, the following node types were not found:
IPAdapterBatch
IPAdapterUnifiedLoader
IPAdapterWeights
IPAdapterNoise
Nodes that have failed to load will show as red on the graph.
I've updated ComfyUI_IPAdapter_plus , deleted and recloned, deleted and redownloaded through manager, and I continue to get the same error each time.
no module named "node helpers" is why it fails to import
Is your ComfyUI up to date? That sometimes messes things up for me. Can try a git pull when inside the ComfyUI folder and after that try to update IPA again.
@@elowine I'll give that a try, I haven't updated in a few weeks
It's amazing！very usefully video,thank you
The best channel for learning comfy.
Hello author, read Embed group ipadpt Where can I download this file
I was the 1337 view. Must be a sign!
(Thanks Matteo, for your great work to the community!)
very good tutorial, thanks for all (:
you are CRAZY(in the good way), OMG
will try this with my drawings🔥
Incredible! ThankU❤
Hi Matteo!
Your new video is so great! I want to ask what is your PC specs (CPU, GPU, RAM)?
Thanks a lot for these videos, I learned a lot!
amd 59xx, 64gb ram, nvidia 4090 running on linux
These videos are great!
Not gonna lie, a lot of this just went straight over my head. lol How the hell did you get so good with comfy?
practice I guess :)
Matt30! Multo-grazie!
I can stop F5-ing now 😄I'm 300 images in, and still no back of the head image, I love the tech, I hate the prompting 😅
use the composition IPadapter
Great update! Banodoco is indeed amazing!
My utmost gratitude man, what you're doing is insane!
Ugghh i love your brain sir ...
I knew it! the zombie apocalypse has started!
Awesome!! I enjoy all of your videos
I love this guy!
Matteo é o melhor!
Looks like Kara from detroid become human :)
Outstanding as usual, thanks for the great work!
This is INCREDIBLE. Thank you!
Wonderfull THX
Fantastic! 🎉 wonderful video
Странно, я полностью повторил это видео и у меня совершенно другой результат. Все модели теже. Но на выходе получается уродливое видео :(
Этот метод больше не работает?
I'm happy thanks to your video. thank you.🥰🥰🥰🥰
you are geniuses
my efficiency-nodes-comfyui have been failing to install. What should I do .I have repeatedly installed it many times
❤
Great as always!!! 🎉
Always amazing!
I LOVE YOUR WORK MAN
Matteo: "Life's too short for slow generations" 😅👍
Cool
🐐🐐🐐🐐🐐🐐
Awesome work
just doing my part
Thank you!
Super cool!! keep going 👍