#### Links from the Video #### Download Workflow: drive.google.com/file/d/17UQXmDRvPLLI7c6g76LlZF4BjMuoRHQa/view?usp=sharing SVD Model Download: huggingface.co/stabilityai/stable-video-diffusion-img2vid/tree/main Enigmatic_e Video: ruclips.net/video/imyQuIiuRnA/видео.html
Please add to this comment that you have to click on RIFE VFI ckpt_name and then select either rife47.pth or rife49.pth There are several comments about people who got an error. thank you.,
This is a good step, as soon as they allow LCM and text input to be added it will be game changing. The stability of the backgrounds is outstanding compared to animated diff but the motion of people is severely lacking as well as no lora support. I have high hopes for this to improve fast however.
Thank you, Olivio! I was starting to doubt my computer's capabilities after my tests failed yesterday. However, after following this tutorial, everything clicked into place. It turns out that the crucial missing piece was the 'update all' step in my tests. Lesson learned!
An important note: In the video you shows svd and svd_image_decoder as 14 fps and 25 fps models, but that's incorrect... The model for 25 fps video is svd_xt, located in another different hugginface link...
in model info it mentioned as "we also finetune the widely used f8-decoder for temporal consistency. For convenience, we additionally provide the model with the standard frame-wise decoder"
I still havent made the switch to comfy. I have a fairly complex workflow in a1111 that I am sure if I spent enough time could automate in comfy , but I like all the tinkering and inpainting and iterative steps that I use in a1111. In comfy I feel like you setup a workflow and its all done in one shot. I guess I could create a workflow for each step and have it work somewhat the same though.
I automated my a1111 workflow with nodejs scripts and using a1111 API, but when I look at comfy it looks like you can do all of that in browser without need for coding. Plus everything new always work in comfy first xD So decided to switch today. Time to say bye bye to a1111
Hi Olivio, thank you so much for this video, its clear step by step, helps someone to follow along and implement it. I have an older GPU - NVIDIA GeForce GTX 1080 Ti, and it takes quite a long time to render the video. How much time does it take on your system?
Thank you for the detail instruction! I was able to create a 512x512 14 frames rocket video finally (Prompt executed in 599.47 seconds) with my rx580 4gb video card.
I know you say "ComfyUI" is the "future" of AI, but I can't bring myself to use it. The learning curve is much steeper than something like A1111. Yes you have massive control and I'm not unfamiliar with Node workflows (Unreal, etc) but in terms of mass marketability and ease of use for the average consumer, it's not the way to go in my opinion.
@@Vestu StableSwarmUI might be something for you (even if it's still alpha). I learned to love comfyUI because it's the only thing running smooth on my potato pc.
I was thinking the same like you, but believe me, after you first try it, you won't ever use Automatic1111 again. It is fun to watch the nodes "working" while rendering and it's very intuitive - even if it doesn't look like it at first glance@@Vestu
@@TPCDAZ With 8GB GPU sdxl doesn't work well in automatic 1111. I've disliked nodes for 20+ years but I just use somebody else's workflow file for comfy. I never need to change the connection wires or make new nodes. I've been using Foocus a lot though recently. So I think Auto1111 is for elites now.(unless using sd1.5 model)
I'm eagerly looking forward to being able to generate my own Warhammer Fantasy Battle movie with nothing but a script I have generated using any random uncensored text-to-text model in a year's time
Amazing work as always Olivio! - If anyone here is getting an "Error occurred when executing KSampler: unsupported operand type(s) for *=: 'int' and 'NoneType'" and has a solution please let me know. Thanks!
Please help, I've gotten these errors before and gave up in the past with other workflows.. I've attempted with my main Comfyui install and a fresh one. I've run the manager and installed all missing nodes, did an update check, and restarted multiple times. When loading the graph, the following node types were not found: ImageOnlyCheckpointLoader SVD_img2vid_Conditioning FreeU_V2 VideoLinearCFGGuidance Nodes that have failed to load will show as red on the graph.
(Anyone else having problems when they try to update all in the comfy ui manager after i update it continuously says restarting) i just got it working great video i do wish yo explained some parts better
Thanks for the helpful video. Do you think there is a enhance detail node I can apply at some point in the workflow that will eliminate the blurriness and disfigurement that happens at times?
I've tried this but for most pictures all I get is a simple animation that could be done much faster and easier with any mobile phone app for video editing, I hope this AI improves soon though
Image complexity doesn't seem to matter as long as you have shallow depth field like a distinguishing difference between the foreground person or object and the background such as a clear border.
Has anyone else noticed that when using this default workflow that the output video appears "smooth" or smeared in comparison to the input image? This even happens when starting with a lower resolution image.
Thanks a lot man. I have a problem: when installing all the missing nodes, all of them install correctly but one: the VHS VideoCombine. I get the same error again and again: "When loading the graph, the following node types were not found: VHS_VideoCombine Nodes that have failed to load will show as red on the graph." I tried to update it, fix it, and even deleting it and reinstalling it, but nothing seems to work for me. Do you know what could it be? Thanks!!
@@MegaGasek Thanks for the response. So you basically have no control. Things will move the way the AI thinks they should. Too bad, I was hoping for a way to generate consistent views of an object/subject from different angles for photogrammetry. It would have been great if you could have fed it 14 viewpoints.
@@c0d3_m0nk3y Yes, I know. However I think this is just the very beginning of great things to come. The way ''AI'' is evolving I think you'll have your wish in no time at all. I do think that processing power will be an issue for more elaborate things.
thank you , I make it work, so now question, how do i manage the animations, right now its like zoomin zoom out ... do there any other workflow where we can add also prompt?
Help me Olivio Wan Kenobi, you are my only hope... No matter image i feed into it, all i get is a slight zoom in. Tried cars, People, Beach, a fighting scene etc. I used the exact workflow from the link without changing any settings. I even scaled down the images in gimp to a 1024 x 576 resolution.
Can't we just add text to image to the workflow then feed that image to SVD? That could be a temporary solution to text to video until we get official support right?
Yeah there's a workflow that uses text to generate an image then it uses that for the video. Find the post titled "ComfyUI Update: Stable Video Diffusion on 8GB vram with 25 frames and more" in the SD subreddit and the link they provide includes the workflow.
I tried to do the steps you have shown, but it seems like i run into some errors... If I try to do the "Update All" it gives me the following error: Failed to update ComfyUI or several extensions / TypeError: Cannot set properties of null (setting 'onclick') If i load the json config it says: When loading the graph, the following node types were not found: RIFE VFI VHS_VideoCombine Seed (rgthree) Nodes that have failed to load will show as red on the graph. How can i fix this?
When loading the graph, the following node types were not found: RIFE VFI Seed (rgthree) VHS_VideoCombine Nodes that have failed to load will show as red on the graph.
I downloaded everything correctly, but my ckpt isn't loading in, despite it being in the right spot. All it gives me is an undefined/null and no option to select any models. I have the paths set up correctly in the YAML file, so I'm not totally sure what the issue is
#### Links from the Video ####
Download Workflow: drive.google.com/file/d/17UQXmDRvPLLI7c6g76LlZF4BjMuoRHQa/view?usp=sharing
SVD Model Download: huggingface.co/stabilityai/stable-video-diffusion-img2vid/tree/main
Enigmatic_e Video: ruclips.net/video/imyQuIiuRnA/видео.html
👋
hey, you forgot the link to the Comfy UI Manager Extention :)
that is so cool
Is this possible to install this locally on PC with window 7???
Please add to this comment that you have to click on RIFE VFI ckpt_name and then select either rife47.pth or rife49.pth
There are several comments about people who got an error. thank you.,
2023 and installs are still crazy. One day, we'll be so advanced, we can click "install" and things will get installed simply... one day.
This is a good step, as soon as they allow LCM and text input to be added it will be game changing. The stability of the backgrounds is outstanding compared to animated diff but the motion of people is severely lacking as well as no lora support. I have high hopes for this to improve fast however.
Thank you, Olivio! I was starting to doubt my computer's capabilities after my tests failed yesterday. However, after following this tutorial, everything clicked into place. It turns out that the crucial missing piece was the 'update all' step in my tests. Lesson learned!
Awesome! Thank you
I'm in the process of making a WWII movie trailer with SVD. So fun.
The age of AI never ending shows we can binge is coming.
Doom Scrollers gonna be doomed 🤣
I'm finding it actually works quite well at different resolutions. I've done 512x768, 768x768 etc and they turned out good.
can it do 1280x720?
@@Steamrick Im running it at 2416x1360
yes, but 576x1024 is the suggested res, because that is what it was trained on. :)
@@OlivioSarikas Yeah definitely. I ended up trying it because at native resolution I couldn't generate 25 frames without running out of vram lol
@@Gh0sty.14 how much vram?
RTX 3060 with 14 frames model took about 2 mins per render ❤❤❤
with 3090 few seconds ;) ... around 10 seconds
4070TI Super around 1 minute for the 25 frames model with a multiplier of 4
rtx 3060ti, it take decates :/ because of my image 1024x1024??
@@MuratAtasoy check if your GPU is running while generating video.... maybe 8Gb VRAM is not sufficient....
Every time you're pronouncing the word "copy" incorrectly, exactly like my boss does 😅love it hah
An important note: In the video you shows svd and svd_image_decoder as 14 fps and 25 fps models, but that's incorrect... The model for 25 fps video is svd_xt, located in another different hugginface link...
so what's the difference then?
Oh my gosh you’re right. I just grabbed the model from the post on the Hugging Face homepage and it was the xt version
in model info it mentioned as "we also finetune the widely used f8-decoder for temporal consistency. For convenience, we additionally provide the model with the standard frame-wise decoder"
Not all heros wear capes...
Thanks for noticing and saying something.
Olivio your attention to detail in sharing these workflows is phenomenal - thank you
Hello, thank you for the tutorial! I'm wondering how I can get the RIFE VF1 and Video Combine nodes?
Hooray! Happened! I figured it out in just 2 days )) Thank you very much!
thanks for the quick and easy breakdown have been putting off learning comfy but this helps a lot
Thank you Olivio. I'm loving comfyUI, specially because it is old PC friendly. I'll give it a go.
comfyUI is like Lego for AI. Makes me feel like a kid in a candy store :)
I hate ComfyUI!!!! Really hate it with my heart!
You not alone.
skill issue
Its great for obsessing over precision pores and moisture on eyelashes but for rapid ideation it stinks.
I hear you bro. I'll wait for an Automatic 1111 or Gradio version
Hate it too!
excited for this, but I hate node based apps. Hopefully there will be an Auto1111 version soon.
Applying emotions to an interface... Reminds me of the days when people fought over Playstations, Xbox and Nintendos. Facepalm.
You don't have to make the node networks yourself, just grab one's that do what you want and get creative. plenty out there
@@fenriswolf-always-forward your comment suggests there's no such thing as a bad interface. lol
I still havent made the switch to comfy. I have a fairly complex workflow in a1111 that I am sure if I spent enough time could automate in comfy , but I like all the tinkering and inpainting and iterative steps that I use in a1111. In comfy I feel like you setup a workflow and its all done in one shot. I guess I could create a workflow for each step and have it work somewhat the same though.
I automated my a1111 workflow with nodejs scripts and using a1111 API, but when I look at comfy it looks like you can do all of that in browser without need for coding. Plus everything new always work in comfy first xD So decided to switch today. Time to say bye bye to a1111
Gonna wait for the A1111 version
Thanks for the guide! Haven't used deforum much lately, and runwayml tends to get pricey. This will be fun to play around with.
you are welcome. i love these new models :)
Hi Olivio, thank you so much for this video, its clear step by step, helps someone to follow along and implement it. I have an older GPU - NVIDIA GeForce GTX 1080 Ti, and it takes quite a long time to render the video. How much time does it take on your system?
Thank you for the detail instruction! I was able to create a 512x512 14 frames rocket video finally (Prompt executed in 599.47 seconds) with my rx580 4gb video card.
Is it possible to add this extension to Stable Diffusion automatic1111 workspace?
Finally! That looks so amazing.
Ha, I was waiting for your take on the install my friend
Kinda cool. Can see it being useful for making mini-clips for presentations or youtube vids (more interesting then putting still images I guess).
With the 24 frame model I got around 6s/it, 14 frame model around 3s/it.
8gb v-ram, rtx3070
Thnks man!
I will return to see in the comment the hardware needed.. no infos on the net..
Thank you very much, Olivio!
You did not mention how much VRAM requires to work ?
12gb
minimum 8GB, 6 if you do smaller image size
for me takes 9 GB 25fps model @@Eleganttf2
I know you say "ComfyUI" is the "future" of AI, but I can't bring myself to use it. The learning curve is much steeper than something like A1111. Yes you have massive control and I'm not unfamiliar with Node workflows (Unreal, etc) but in terms of mass marketability and ease of use for the average consumer, it's not the way to go in my opinion.
Or it should have some Easy mode UI with most common building blocks and you could enter the current "advanced" mode
@@Vestu StableSwarmUI might be something for you (even if it's still alpha). I learned to love comfyUI because it's the only thing running smooth on my potato pc.
I was thinking the same like you, but believe me, after you first try it, you won't ever use Automatic1111 again. It is fun to watch the nodes "working" while rendering and it's very intuitive - even if it doesn't look like it at first glance@@Vestu
@@franciscodurand5209 it's as fun as glaring at a cluttered house that is long overdue tidying.
@@TPCDAZ With 8GB GPU sdxl doesn't work well in automatic 1111. I've disliked nodes for 20+ years but I just use somebody else's workflow file for comfy.
I never need to change the connection wires or make new nodes. I've been using Foocus a lot though recently.
So I think Auto1111 is for elites now.(unless using sd1.5 model)
oh dang, will have to try it now! thanks for the sources! :)
I'm eagerly looking forward to being able to generate my own Warhammer Fantasy Battle movie with nothing but a script I have generated using any random uncensored text-to-text model in a year's time
Thank you very much, awesome tutorial :)
Do anybody know how much VRam you need to run this?
You can run it on 6Gb of VRAM
24 fps model takes 9 GB for me @@MyWhyAI
I thought you need 40 GB of VRAM for SD Video to work? How much VRAM do you have, Olivio?
For me takes 9 GB of 24 GB - 24 fps model ( rtx 3090 )
3050 8 gb vram, 16 ram gb r7 1800x and take me 3 min to create a vid@@mirek190
Amazing work as always Olivio! - If anyone here is getting an "Error occurred when executing KSampler: unsupported operand type(s) for *=: 'int' and 'NoneType'" and has a solution please let me know. Thanks!
If you disable FreeU Advanced, problem goes away.
It works! Thank you@@GBUK666Thank you!!! You legend!!!
Do you mean to delete the FreeU node? I did that and still same error. Odd@@GBUK666
Thank you for the information.
can't wait to get some free time to check this out
Did not see the link to the comfyui manager extension. : ) thanks for the vid
What's the minimum requirements in regard to VRAM, RAM, graphics card, etc.?
Just did the beach vid. on rtx2070 8gb, took 20min. My card is underclocked 10%. Used the 25fps model listed on Enigmatic_e Video.
Please help, I've gotten these errors before and gave up in the past with other workflows.. I've attempted with my main Comfyui install and a fresh one.
I've run the manager and installed all missing nodes, did an update check, and restarted multiple times.
When loading the graph, the following node types were not found:
ImageOnlyCheckpointLoader
SVD_img2vid_Conditioning
FreeU_V2
VideoLinearCFGGuidance
Nodes that have failed to load will show as red on the graph.
im having the same problem! when i check for missing nodes nothing shows up :(
Unfortunately doesn’t work on my M2 Mac MBP. Getting “RuntimeError: Conv3D is not supported on MPS”
x3 ahah mac
You jumped from the SVD to having it open with no-information on pathing or troubleshooting. Totally hopeful.
thank you Oliviio
Anyone know if there is a way to choose which direction the resulting video pans left vs right?
but what did comfui do that the others didnt here...
Awesome video! As someone else stated below, the SVD-XT model is for the larger FPS count though. Otherwise, perfect video :D
thanks for the guide!
thank you for the tutorial it help a lot
(Anyone else having problems when they try to update all in the comfy ui manager after i update it continuously says restarting) i just got it working great video i do wish yo explained some parts better
Thanks for sharing 💗
Thanks for the helpful video. Do you think there is a enhance detail node I can apply at some point in the workflow that will eliminate the blurriness and disfigurement that happens at times?
Thanks for being so clear. You're all about ComfyUI. I don't need to subscribe to you anymore.
Looks like those glasses were added with stable video diffusion. Easy to tell.
Thanks for the nice tutorial - i have one question - in which node i can change the length of the final video? Best Felix
Awesome! So helpful! :) :) :)
I love comfy u.i.
Cool thank you !
I've tried this but for most pictures all I get is a simple animation that could be done much faster and easier with any mobile phone app for video editing, I hope this AI improves soon though
Thank you
thanks Tyson fury
Comfy ui is awesome. I am used to nodes in DaVinci resolve so perfect.
Image complexity doesn't seem to matter as long as you have shallow depth field like a distinguishing difference between the foreground person or object and the background such as a clear border.
true, but what i meant to say is that it won't do a complex dance move or anything like what you see in the deforum and kaiber videos
Thanks!
Thanks Olivio, and don't worry, many of us can handle both A1111 and Comfyui. Keep em coming. 24GB Gang Gang-
Has anyone else noticed that when using this default workflow that the output video appears "smooth" or smeared in comparison to the input image? This even happens when starting with a lower resolution image.
👍Cool as ever
Thanks a lot man. I have a problem: when installing all the missing nodes, all of them install correctly but one: the VHS VideoCombine. I get the same error again and again:
"When loading the graph, the following node types were not found:
VHS_VideoCombine
Nodes that have failed to load will show as red on the graph."
I tried to update it, fix it, and even deleting it and reinstalling it, but nothing seems to work for me.
Do you know what could it be? Thanks!!
my favourite way is not with comfy.....
Everywhere I go it seems that SD checkpoints, Loras or ControlNets aren't named in any usefull or logical way.
Very nice
thanks man... !
Is there a way to extend a video after it has been created?
Do you only provide 1 picture and it generates 14 frames or do you provide 14 frames and it will interpolate between them?
It will generate the frames from one picture. I think that if you couple it with Topaz Video AI/Handbrake you'll be able to do amazing stuff.
@@MegaGasek Thanks for the response. So you basically have no control. Things will move the way the AI thinks they should. Too bad, I was hoping for a way to generate consistent views of an object/subject from different angles for photogrammetry. It would have been great if you could have fed it 14 viewpoints.
@@c0d3_m0nk3y Yes, I know. However I think this is just the very beginning of great things to come. The way ''AI'' is evolving I think you'll have your wish in no time at all. I do think that processing power will be an issue for more elaborate things.
its like installation on nuke boom
what version of Comfy UI are you using? mine dont have that fancy share button
update it
Thanks
ComfyUI sucks. It can't copy seed from result image when batch size is greater than 1. Like how do you people even use it ?
ill wait for the web gui lol
Now you can create videos with SVD! Or can you? (suddenly Jake Chudnow - Moon Men Instrumental starts to play ) :D
Would it work on a mac mini ? considering buying one…
thx for the video! tried it for a few hours, but without any other modifications it's nothing for me... (like length, prompts, maybe loras)
Do we have to install ALL the node?
Does not work with M1-M2 silicon macs. Get an error.
Hello. I can't configure it. Where are the downloaded models placed within ComlyUI? In what folder, or how are they installed?
thank you , I make it work, so now question, how do i manage the animations, right now its like zoomin zoom out ... do there any other workflow where we can add also prompt?
Sadly can’t get this to work with AMD gpu in windows
Imagine going through all this for a 14 frame GIF.
You are the definition of spoiled.
cuckold
When loading the graph, the following node types were not found:
VHS_VideoCombine
How do I create longer videos?
Help me Olivio Wan Kenobi, you are my only hope...
No matter image i feed into it, all i get is a slight zoom in. Tried cars, People, Beach, a fighting scene etc. I used the exact workflow from the link without changing any settings. I even scaled down the images in gimp to a 1024 x 576 resolution.
Can't we just add text to image to the workflow then feed that image to SVD? That could be a temporary solution to text to video until we get official support right?
Yeah there's a workflow that uses text to generate an image then it uses that for the video. Find the post titled "ComfyUI Update: Stable Video Diffusion on 8GB vram with 25 frames and more" in the SD subreddit and the link they provide includes the workflow.
Is there any specific PC requirements to run this?
what parameter changes the length of the video?
x2
I tried to do the steps you have shown, but it seems like i run into some errors...
If I try to do the "Update All" it gives me the following error:
Failed to update ComfyUI or several extensions / TypeError: Cannot set properties of null (setting 'onclick')
If i load the json config it says:
When loading the graph, the following node types were not found:
RIFE VFI
VHS_VideoCombine
Seed (rgthree)
Nodes that have failed to load will show as red on the graph.
How can i fix this?
When loading the graph, the following node types were not found:
RIFE VFI
Seed (rgthree)
VHS_VideoCombine
Nodes that have failed to load will show as red on the graph.
You gotta click on the one already there and change it to RIFE 47 or 49.
Hey - where is the ComyUI instuction? you said you where going to link?
Why I do not have access to the preview video? I need to click on "Queue Prompt" and render it to see
I downloaded everything correctly, but my ckpt isn't loading in, despite it being in the right spot. All it gives me is an undefined/null and no option to select any models. I have the paths set up correctly in the YAML file, so I'm not totally sure what the issue is