NEXT-GEN NEW IMG2IMG In Stable Diffusion! This Is TRULY INCREDIBLE!
HTML-код
- Опубликовано: 13 фев 2023
- ControlNet is a brand new neural network structure that allows, via the use of different special models, to create image maps from any images and using these informations to transfer data from one image to another, making it a really powerful option in the image-to-image tab inside Stable Diffusion! This gives you even more power and more control over the final result than before. So in this video, I will show you how to install the extension and how to use the differents models to get the best results possible!
Did you manage to install the extension? Let me know in the comments!
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
SOCIAL MEDIA LINKS!
✨ Support my work on Patreon: / aitrepreneur
⚔️ Join the Discord server: bit.ly/aitdiscord
🧠 My Second Channel THE MAKER LAIR: bit.ly/themakerlair
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Runpod: bit.ly/runpodAi
Miro board: miro.com/app/board/uXjVPnB9L2...
Extension URL: github.com/Mikubill/sd-webui-...
The models: huggingface.co/lllyasviel/Con...
Special thanks to Royal Emperor:
- BSM
Thank you so much for your support on Patreon! You are truly a glory to behold! Your generosity is immense, and it means the world to me. Thank you for helping me keep the lights on and the content flowing. Thank you very much!
#stablediffusion #controlnet #stablediffusiontutorial
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
WATCH MY MOST POPULAR VIDEOS:
RECOMMENDED WATCHING - My "Stable Diffusion" Playlist:
►► bit.ly/stablediffusion
RECOMMENDED WATCHING - My "Tutorial" Playlist:
►► bit.ly/TuTPlaylist
Disclosure: Bear in mind that some of the links in this post are affiliate links and if you go through them to make a purchase I will earn a commission. Keep in mind that I link these companies and their products because of their quality and not because of the commission I receive from your purchases. The decision is yours, and whether or not you decide to buy something is completely up to you.
![Dimension 20: Never Stop Blowing Up Trailer [Dropout Exclusive Series]](http://i.ytimg.com/vi/xYeXpbHnGEM/mqdefault.jpg)
This extension is amazing! I'm having an absolute blast with it. It is solving so many of my problems with matching the look of realism photos with what I'm feeding it for an image and with my prompt. Thank you so much for what you do. You've been my first go to on RUclips for weeks!
I'm having fun with the sketch pre processor. Runs fine on my gpu (1060GTX 6GB), not even using the --lowvram option.
Not even week goes by and there's image to image significant enhancement. What a time to be alive!
Saw info about this on Reddit, I knew you would put out a how-to video so I waited to install. Glad I did, you did not disappoint. Sincerely, your loyal subject.
This is exactly what I need. Last night I was having fun with the blend option in Midjourney and wished SD has something similar. This video came at the perfect time
Yet again, you have manged to blow my mind! Thank you for showing this new amazing functionality! Feels like these tools are getting more insane every single day.
The pose one looks great, you can pose a character in a 3d program and use it as a base.
What if you use your own body to pose?
Perhaps it will be useful with some simplified solutions like Daz 3d
Or with poses library
Yup. I've been using Daz to help me with comic book character poses for years and it took only some seconds to put a character in different poses (if a person says it takes more than that, that's because they're newcomers or have not much experience with 3D programs).
and yeah, the 3D software thing is actually a good idea if you implement a good inverse cinematic thingy on it. It can probably save time and be faster than a google search for simple (or really specific and complex) poses. I've been learning how to do that last night in blender, I'm almost done with the model, it's actually not too hard to make.
You don't even have to render anything btw, you just have to take a screenshot of it when the angle and everything seems okay, and then paste that in the webui. Works like a charm.
Exactly
HELLO HUMANS! Thank you for watching & do NOT forget to LIKE and SUBSCRIBE For More Ai Updates. Thx
Do we know if these are safe? I don't even know how to scan pickle anymore. I have avoided them for months. Amazing video by the way. Thank you.
“Type cimdy”
👍👍👍👍👍👍👍 Great extension! paused the video, got it all downloaded and installed before I finished the video. Runs fine on my 1060GTX 6GB, even without the lowvram option. Actually 10x faster without it. only the xformers option is needed.
@@peterbelanger4094 If you can Peter? I have a RTX 4080 16GB. I've never used any Xformers. Should I look into them? And if so what purpose they server? lol If you can't explain that is fine.
what SD version are you using? I only have..1.5 I think and the extension only appears in txt2img, not in img2img
downloading the models now and will try. thank you so much. this seems very powerful. In fact, I've been spending more time on img2img lately. even at the current state, it is fantastic... can't imagine the possibilities with this new extension
Installed it straight away and can honestly say I'm super impressed 👍👍👍
Wow. So exciting. Thank you dear Overlord 💪🙏🏽
It's epic, power of stable diffusion is upgraded ! ❤️
My biggest complaint about SD has been the lack of control.
In order to make comic books and alike you need to be able to precisely control the pose of the characters.
Gonna see how well this holds up.
Thank you for the video!
Top notch training! Thanks for this comprehensive overview! Looking forward to testing this out!
Installed it just couple of hours before a vid. Thx for a tut!
Thanks for being on the bleeding edge of this and imparting your knowledge for artists like me. My question is, what is the best way to implement Stable Diffusion in Blender. I used a version like 3 months ago and now that seems so outdated.
Thank you for the wonderful walk through on this new extension. I have a lot of 2d pose sheets/sketches from various artists and I can totally see using them with a controlnet.
Simply amazing, thanks for the video! This makes img2img even better than I could have imagined. .
magic... so much more to come thank you for keeping us on the leading edge..
I've been wishing for this since day one of using SD! This is going to be so useful for generating versions of my graphic design work!
i saw people talking about this concept on several other channels, and all were like "i hope this comes out for auto1111", and i'm all "it is!" - and linked this video.
hopefully you get more subs.
Amazing tool-
solves a lot of RNG for us to get closer to the designs we are looking for
ty again Aitrepreneur for helping get my home studio going
This looks like a lot of fun and really good for professional use. Thanks for sharing dear AI overlord
This just keeps getting better and better! Thanks for putting this together. Is there any way to merge two different images? Like take the pose from one image and implement it in the style or background of another?
its called inpaint. just use photoshop plugin for this.
paint over (or put image) on different layer and click inpaint
This is insanely great! Thanks for sharing this info.
If this works as well as it appears to, this is both game-changing and life-changing for artists like myself that work in 3D but want more illustrated or toony results. I've waited 24 years to be able to make true 2D art with 3D methods. Even if it's not perfect yet, this gives me hope.
I'm playing around with the scribble model and i'm absolutely blown away!
topaz studio 2 was coming close but THIS is next level
I had the same thought. Makes me want to train my cc4 characters
@@PriestessOfDada I've had decent luck using untextured DAZ figures as ControlNet pose references, but I do not know what the results would be if you train a checkpoint or LoRA on a complete 3D character. If you can still get 2D or anime results from it...well, I'll have a lot training to do. ;)
this is pure gold
Cool 😮 A.I community keep surprising us every day
Awesome thanks for sharing! Btw, if you click the down arrow just to the right of the letters LFS it just downloads the models without opening another tab. Small but useful tip :)
Thanks a lot for sharing your miro boards, those are normaly test that I would do myself to understand how the system works, you are saving us a lot of tech investigation time! :)
I really like txt2img but img2img has been where I've spent most of my time. Scribble mode looks amazing! I need it in my life.
Thanks again for good contents! Pose thing looks interesting 👀
I recently experimented with blue willow and I'm really amazed to use it. Blue Willow recently launched V2 with a brand new model upgrade. Now I got better-quality images with more aesthetic outputs
I've been waiting por the 1.5 depth model and now it's here, and with a few friends 🤗🤗
Nice, comprehensive video -- and thanks for the Miro board in particular! Makes me think there's great potential for AI workflows in there. :)
I'm droolling at the thought of using the pose model!
My favorite youtuber 🧡
Thanks for another great vid! I'm more interested in the open pose model because then you won't need to prompt a pose too much. Seems like in the video it can retain some details like the clothes color so it also needs a prompt for it to change. Very interesting.
Edit: Some more interesting things
- It can accept characters provided the model knows them (Azur Lane characters for example)
- Like from the video it does eat up vram fast, my Linux almost crashed one time lol.
Awesome, thank you!
Absolutely Crazy. thanks!
This is absolutely amazing, my only complaint is that it doesn’t seem to work with batch img2img. If that gets to working, then this could easily solve the issue with stable diffusion videos where details tend to be “sticky” due to the seed not shifting with the video. This could help stabilize it.
Edit, after an update, it works with batch img2img and it does exactly what I wanted. What a time to be alive!
yoo do you make videos in stable? because i do and I m just interested in batchmode /animation. consistens characters and places. can we connect on insta ?
This is VERY impressive
Great breakdown
this is awesome
this is truly nextlvl stuff. im glad i found this video. has anyone already tried to create a depthmap with controlnet and then using that to create a 3model in blender? there's some good tutorials on here as well, you might wanna check that out :)
Why have I only heard about this for the first time today!? Wow!!
Never knew you could type cmd on the top directory... thank you so much
Thank you so much.
so cool! I love ya!
thanks so much mate
Wooow, i like this new tool!!
thank you so much 🙂
Thank you sensei bro 🙏
Brilliant. Any tips to install on M1 Mac?
Very Nice!
Dear AIOverlord: Thank you as always for being ahead of the pack, and showing me something useful and amazing. In every example, your two images (input above and input below) are the same ~ What happens if they are different? (Naturally, I couldn't possibly just try it lol)
Big Thanks
I love the way you say "down below"
This is awesome. But I'm kinda confused about how the 3 "inputs" relate to each other. Perhaps this is me just not understanding img2img. Basically my question is how do the prompt, image, and 2nd image (for depth map) relate to each other for the resulting image? Doesn't this new extension mean that we could (also) use only an image or prompt with the image for depth map to generate an image? I would love an in-depth answer 🙏
Goddamn, the webui just keeps getting more and more powerful. 😯
I finished my last short with the old img2img. Downloading all the models and lets see how this goes.
NO WAY THIS IS INSANE
Ooooooo! Very useful
i need this video on a flash drive for reference for ever
Hi and thanks for your amazing guides. I followed thru the steps but for some reason i'm only having access to controlnet-m2m and can't find any other options. Is there any other requisites before install the controlnet extension or something to configure??
This is TRULY INCREDIBLE!
Hello, I have a question , how to find the control stable diffusion with human pose tab ? Thank you for your share.
cannot wait for this to be an extension to txt2img also : prompt for a specific character and then add a scribble or pre process image to the script to get the described character in the pose you want.
It already has this.
@@ClanBez Open the txt2img tab with the extension this video explains installed. At the bottom of the tab you can use the ControlNet just like in img2img, including the scribble model. By providing a prompt and a scribble, you can generate images with lots of control. I suggest lowering the Weight to 0.25-0.5 to start with as you ca get some weird results depending on your drawing skills otherwise. Good luck.
for some reason it only shows up in txt2img for me lmoa
Are you able to create an updated installation video? I've been out of the loop for a few months with Stable Diffusion and feel lost with all the updates.......... :(
This feature finally able to make new image with exact style from original input picture.
Another useful video, cheers. I have a question I was wondering if you might know the answer to. I want to train some models (loras probably) to recognise some designer clothing. So it would be a bunch of items in a particular category. Such as Shirts, Long sleeve/Short sleeve/distressed, Nero collar etc, but I'm not sure which method is best and If I should fully describe the source images or leave out some parts of the caption to get best results, Can you make any recommendations ?
I would suggest using around 50 to 200 images of whatever you're trying to generate, and see how it does with such a dataset. Then, iterate over it untill you're happy with the lora you made.
This is cool. I was wondering could this be used to make line art? That would be a time saver with poses, etc.
I will definitely try it. I wonder if it works with multi characters
The open pose is a game changer, it would be great for video input as well, for things such as dances or movement, is that doable? Seems like doing this in batch would take ages. Maybe this could be implemented within Deforum?
So a few thoughts playing with this. I specifically zeroed in on the pose model because one of the things I've been trying (and failing) to do for a long time now is mix my own photography with generation, but having any sort of control over posing was nearly impossible. Until now!
First off, this works in txt2img as well, so you can supply it a pose reference image and then do your normal txt2img prompts and get a completely new generation in that pose. Mind fracking blown! That said, temper expectations at least for now as the pose estimation is not so accurate. This is rather surprising actually given 2d pose estimation has been a pretty well solved problem for longer than SD has been a popular thing, so not sure whats up there.
Still it has already allowed me to start making fusions of my studio photography with SD generations and it is amazing!
Are you saying this extension can also be used over at txt2img?
@@ysy69 Yes! That's mostly where I am using it right now actually. I'm talking some of my models that i've shot in studio, bringing them into SD in txt2img, putting that photo in for controlnet pose, and then using the txt2img to generate a completely new person in roughly the same pose as the one I photographed.
@@ysy69 txt2img is where it works, not img2img
According to github it should be possible with the open pose model to control the skeleton but gradio isnt easy to work with. I'm sure in a few months we'll have a lot more control.
These tools are insane so far!
@@JohnVanderbeck when you say model, you're referring to SD custom models right? and not people as models? When you bring a photo to SD, that is img2img... in text2img, one doesn't use an image as reference, so I guess you meant img2img and then using the prompt to make the change into a new person, correct?
Finally! If you can say that for something that didn't take 24h 😃
Thank you so much for making this video. I was just talking about how this stuff will solve a lot of posing problems we have without it. Can’t wait to try it with my model photography as references 👏🏻
amazing; but how do we use the batch option? if I wanted to generate 100, we have to drag the input images 100 times?
This is great, the hand one seemed to work for me, at least on one base image. It refuses to work at all with waifu diffusion 1.4e2 though.
i had to restart the whole thing to get the models part to show up, but it did work. too late to test it out now, though: )
Great content! thanks for all the work. If i may give one suggestion. I often find myself coming back to a video to checkout the process again. But I noticed like in this video for example it does not mention Controlnet in the thumbnail or title. Its difficult to look for a specific video in all your videos :) Thanks again.
amazing
Is there way to use img to img and control net to move an existing character, with clothing and style, into a new pose/new position, and have it still be that initial character? (WIthout a Lora, only going off of one image)?
That's amazing! Will it work on Colab as well?
does it work on colab?
Does it work with embeddings? Wood be cool to have consistent characters using this method
Do the measurement at 2:24 always show like that? We can take the OpenPose Wirefigure and make anything we can imagine?
Under 'Preprocessor' I have 'depth' but not 'Midas'. Is that a problem or can I use that one with the control_sd15_depth model?
can we use this for Batch ??
Thanks.
I'd love to see a video/cartoon where you transition from one scene to another by increasing midas weight slowly. Would be a very trippy/dream like way to transition to a totally different place.
Hi! Just wanna ask if this is possible. We have multiple portraits (pre-made not AI generated), I want to use my customers' cat or dog or pets image to transfer it or use it as a head on my pre-made portraits.
Hello Robot, how to colorize an old photo in greyscale or black and white with controlnet? which options or parameters are better? Thanks
Hi. Been trying to figure this out since yesterday. The problem I'm having is enabling the Extension tab. I git pull to make sure everything is updated, but that still didn't work. Do I need to enable it by adding something into the command line?
Man control net open pose + VR FBT with 11pt can seriously open up some possibilities.
K you forgot to copy the checkpoints in the ControlNet/annotator/ckpts if you do then the hand pose works amazingly using preprocessor openpose_hand and the model openpose, thnk you for this amazing video, this changes a lot how we create images
where are those? I don't see them on the github page
@@rmb6037 on the models link page go back once into ControlNet and then enter the annotator/ckpts folder
@@rmb6037 look within "annotator" just above the models folder. Then ckpts into those files.
That is where I'm lost at now. We would those files (pth & pt) go? Just into the model folder with everything else?
@@ThisOrThat13 i noticed they were auto downloading but wasn't working for me and i put them all in here \extensions\sd-webui-controlnet\annotator\ckpts
is there anyway we can use controlnet to batch a bunch of img2img png sequence? keep getting an error trying to save past the 1st control net image generated.
Something you could have covered in the video, is the inpainting using this method. I noticed in their site that you can inpaint a part of an image. I wish you explained that.
Can I install this on my macbook M1Max ?
There exists a safetensors version of the models that take up *signifigantly* less space! but other than that, golden.
Amazing videos! Thank you. Just a suggestion, maybe the part where you download models in your vids should be first, so they download while we do the other steps. I already know your videos so I get ahead of this. But again, thank you.
nice!
how do you use scribble mode for coloring books?