ULTIMATE CONTROLNET Starter Guide!
HTML-код
- Опубликовано: 27 май 2024
- This guide is your ultimate primer for everything ControlNet. ControlNet is the most popular and useful tool that can be used with Stable Diffusion, so in this video I run you through installing the ControlNet Extension
in Automatic1111, how to install all the ControlNet Models you need including the new SDXL Community ControlNet Models, as well as all the compatible T2I Adapters.
I go through every single model and show you what they do and what they're good for. So if you every wondered what the Depth setting was for, or what the SEG model output means, you're about to find out.
🔗 Links: 🔗
--------------------
SDXL Community ControlNet Models: huggingface.co/lllyasviel/sd_...
SD 1.5 Control net models: huggingface.co/lllyasviel/Con...
T2ia Adapters: huggingface.co/TencentARC/T2I...
Yaml files github.com/Mikubill/sd-webui-...
Color map for ufade20k & ofade20k: docs.google.com/spreadsheets/...
📖Chapters:📖
--------------
00:00 START
00:27 INSTALLING CONTROLNET IN A1111
01:41 UNIVERSAL CONTROLNET SETTINGS
02:34 CONTROL WEIGHT
03:01 CONTROL STEPS
03:18 RESIZE MODE
03:50 PREPROCESSORS AND MODELS
04:23 CONTROLNET CANNY
04:42 CONTROLNET DEPTH
05:07 CONTROLNET NORMAL
05:31 CONTROLNET OPEN POSE
06:02 CONTROLNET MLSD (MOBILE LINE SEGMENT DETECTION)
06:20 CONTROLNET LINEART
06:42 CONTROLNET SOFT EDGE
07:00 CONTROLNET SCRIBBLE
07:26 CONTROLNET SEG (SEGMENTATION)
07:58 CONTROLNET SHUFFLE
08:09 CONTROLNET TILE
08:55 CONTROLNET INPAINT
09:49 IP2P (INSTRUCT PIX2PIX)
10:11 CONTROLNET REFERENCE
10:24 T2IA
10:52 CONTROLNET UNITS
11:13 SWEET OUTRO
----------
AI Voice Credit:
ElevenLabs: elevenlabs.io/ 
Наука
This has to be the clearest and most comprehensive descriptive tutorial of control net that I have seen. Again, your videos are top-notch! Saving these to rewatch and sharing these with my friends getting started
The clarity at which you present this is spectacular. You're gonna grow man.
dude is the best at what he is doing right now on youtube, earned a sub bro
Thanks so much!
This was one of the best explanations for control net models I've watched. Concise but still very informative. Thank you my guy
By far the best explanations in the field, can we expect more from you?
I am going to be re-watching this for the foreseeable future. Thank you
You're welcome, gald you found it useful!
Hands down, you're awesome. 5 videos, Top notch quality content. FOREVER A SUSCRIBER 🥰
Thank you, I really Appreciate that! :D
So much I learned from this. Please talk in future about IP-Adapters from control net. The last button in the Controlnet. Maybe you need a separate video for that.
Glad you liked it! That's the plan, the update adding IP-Adapters dropped while I was finishing this video, so expect an update soon!
You crammed all of that into an 11 min video. Best explanation of controlnet I have seen! I learned a ton of new stuff. I will be returning to this video whenever I need a refresher. Subbed!
Exactly what I was looking for. Thanks for sharing.
Awesome!! Been waiting for this.
Probably the best tutorial i've seen for CN ! Finally a clear tutorial I can recommend to my subscribers that asks about controlnet :)
By the way, I was thinking maybe we could colab together on some video subject. I'm trying to gather some SD content creator and create a community network, like back in the days with old YT.
For sure the best one out there
Thanks so much :D I'm still finding my way here on YT at the moment, so I'm just focusing on building up some content first, but defintely something I'd be interested in down the road!
You're already killing it ! Our Editing Style is quite similar, packing up info in condensed manner. And I like your sense of humor ;)@@InciteAI
Hands down one of the best videos on Controlnet. Mate! You should make more videos. Your content is really good. A longer version of this video could include multiple image examples for each one + further drill down into multiple controlnets with image examples.
Long forom content is much appreciated. Thanks again for putting this together.
Great stuff ! Very comprehensive and humorous. Keep up the great work !
this is THE BEST video about control net. THANK YOU VERY MUCH!!!!!
superb video. thank you
Thank you very much for your work! It is extremely helpful. I don't even know how to thank you enough!
Incredible guide, thanks
No proble, Glad it was helpful!
Why did you stop making new video? All your videos here very detailed, easy to learn. You should do more.
Great tutorial, keep up w the good work
I've read that the control net preview was actually only a very close approximation of what controlnet is doing, hence why there was a checkbox option "preview as input" in case you wanted. It's not there in A1111 1.6, so maybe its accurate now.
Love your videos, great forensic dives.
Can you do one on doing animated stuff in A1111?
Finally SD upscale Mask Blur, WORKS in the newest 1111 update,
Awesome!! thank you!! btw can you show us how to implement this function in ComfyUI.. and how to setup MuseV and MuseTalk in ComfyUI as well.. thank you!!
Perfect.
Awesome
So if i just have A1111 can i download SD 1.5 models and SDXL models???
does it matter which checkpoint i use for this?
Every time I try to use OpenPose (SDXL 1.0, A1111 1.6.0, CN 1.1.407) I usually get an error very similar to this: "RuntimeError: mat1 and mat2 shapes cannot be multiplied (154x2048 and 768x320)" How do I overcome this? What does it even mean?
I know this error was common when people tried using the Controlnet models with SD2.1. The standard models are trained on SD1.5 and don't play nice with other models. If you're using an XL model, I would double check that you've selected an openpose xl control net model (huggingface.co/lllyasviel/sd_control_collection/tree/main). I haven't had much time to test these out yet, so I'm not that familiar with them yet.
@@InciteAI Thanks, I was trying to use the SDXL 1.0 model, but as soon as I switched to the 1.5, or a model trained on 1.5 I was fine. Thank you though!!
It's trying to multiply two matrices that are not matching sizes. Basically the two models do not connect to each other. Like trying to plug a USB into a HDMI slot.
Love your content. Thank you for all the work you put out.
Been having problems running some Preprocessors in ControlNet a1111 and hoping you can help me.
I've tried updating, deleting and reinstalling, I've tried running SD a1111 in different browsers, redownloading the models, tried etc., and I just keep getting the following error:
"
urllib.error.URLError:
"
Anything come to mind that could help me solve this?
Like I mentioned, only a few preprocessors work.
For example, for "Depth," only the "depth_zoe" works, all others give the error. For "OpenPose", none of the preprocessors work. This happens both in SDXL and in 1.5 models. "Canny" on the other hand, they all work.
Any ideas would be great. Thank you in advance.
Hey there! That's a strange one for sure, and I haven't seen it before, but the error you're seeing is a network related error, and since I assume it's running locally, something might be interfering with the localhost. Might be worth checking your firewall or antivirus settings to see if they are interfering at all? I would check out anything you have running that may affect local networking, VPN etc. These are all uneducated guesses of course 😅 You could also try a fresh install of python if all else fails.
Hope you figure it out!
Hi, @@InciteAI
Thank you for taking hte time to respond.
Hi, @BitesizedGenius
Was actually was able to figure it out with the help of Chat GPT!
Looking at the entire read-out in the Command Line there was actual paths to the missing Annotators for each of the Preprocessors.
So, while I'm not actually sure what caused the underlying problem, I was able to download the missing files manually, and it seems to be working.
Thank you again!
A question
what ias the difference between the t2iadapter and the control_v11 files ?
Well for the most part, they are 2 different ways to acheive similar outcomes. ControlNet and T2l use different network structure modification to control the image generation, so basically they impose themselves on the new generation slightly differently. It's thought that T2I adapters are faster and more specific, but ControlNet is more robust and detailed. (I should have put that in the video xD)
@@InciteAI Thanks you very much. Great video 👍🏾
Love the opening. Funny stuff
Thanks :D, I wanted to channel my inner David Attenborough xD
I'm just getting error after error... is all this irrelevant with the advent of XL models?
I find your videos very very detailed and helps with small things al the other channels don't talk about. Keep doing your thing! You deserve much more followers than what you got. Hope you blow up. Now that I've kissed your @$$. Can you tell me what the Preprocessor Resolution does? LOL
Thanks! I really Appreciate that! My understanding is that this adjusts the resolution of the controlnet map that's created, so a higher resolution would give you a more detailed map. This costs more VRAM though, so it might cause issues or just take longer to generate an image if you're tight on VRAM!
too much info , but it is nice explanation , waiting to to see an examples tutorials for most of these control net models
Yeah, ControlNet is pretty expansive, but I wanted to put together a worthy introduction. Definitely more to delve into here!
I thought you had a pretty good balance of detail tbf
How to get rid of bad fingers?
There's lots of things to try, openpose hands or inpainting comes to mind. There also som LORAs out there trained for this.
@@InciteAI honestly I didn't find any loras about fix fingers. they're all about models like Emma Watson or somebody else
LOVE THIS VIDEO ON CONTROLNET! Like and sub from me!
Thank you for breaking it all down in plain English for morons like me. It's hard to find AI tutorials that don't assume tons of prior knowledge.
The video content is very good. But it's too fast. Thumbs down, sorry.
Play at slower speed.
@@crobinso2010 I have made. And it was still too fast.