hey can you do a video on ANYDOOR, please ? I read their github and it seems such a cool tool to change inject location and stuff, but sadly the GRADIO DEMO only has inject transfer from one image to another. Can you make a tutorial on how to use all the other tools that Anydoor provides ?
I wasn't able to get it running. I keep getting a ton of errors, like "OSError: Error no file named diffusion_pytorch_model.bin found in directory data/ckpt/segmentation_mask_brushnet_ckpt." But that file ends in .safetensors not .bin
Careful with those puns, Sebastian Kamph will become jealous. 🙂 The results look great but the one time I tried Anaconda it didn't work and seemed to be more of a mess. Then the hassle with the models so I think I'll wait until it hits Forge, which shouldn't be too long. Cheers NR.
@@NerdyRodent That's cool, it may have been something my end like a misunderstanding on my part. I also realised after that it might have been Miniconda, not Anaconda that I was using.
I wonder what is more powerful, the IOPaint (that I noticed through your video last month), or this BrushNet. Evaluating after having seen both of your videos, I would tend to rather use IOPaint. But I am sure you must have compared both, or at least have a much better insight?
@@NerdyRodent maybe we can expect some video from you about how to use this "PowerPaint" feature you described? Also a comparison of different models and how you use them in IOPaint would be very interesting. I cannot find a lot of input on how IOPaint can be power-used...
@NerdyRodent I completed every steps without any issue for installation, but in the end, the local url is unavailable, it fails to connect to it, any idea why ?
@@NerdyRodent What'd'ya mean by extra security ? I tried with temporarily disabled firewall, checked my browser settings and flushed my DNS, that changed nothing wether I use Chrome, firefox or Edge. Now maybe there is a security setting within Brrushnet that I'm not aware of, if that is the case please could you tell ?
If you’ve not got any extra security running, it could be that you’re using Microsoft Windows which will fail on basic routing tasks. If that’s your case, you’ll need to use localhost as the address instead!
I think your differential diffusion workflow doesn't use its full potential for its blending capabilities. Ideally after segemting the bear you could expand the mask and blur the expanded area so you get a gradient arround the bear. (or even blur without expanding to give it a bit of bear to change) Depending on the blur parameters and the added padding, your results should be much better. The differential diffusion allows you to use non binary masks and let the gray level define the allowed changes afaik.
@@NerdyRodent Could you help me with this if you can please? I tried to find out what is the problem but nothing was found :( I have all the data structure and checkpoints and have the "sam_vit_h_4b8939.pth" but I don't know why this happens when I run the app_brushnet.py. Traceback (most recent call last): File "D:\AI-Programs\BrushNet\examples\brushnet\app_brushnet.py", line 13, in mobile_sam = sam_model_registry['vit_h'] (checkpoint='data/ckpt/sam_vit_h_4b8939.pth').to("cuda") File "C:\ProgramData\miniconda3\Lib\site-packages\segment_anything\build_sam.py", line 15, in build_sam_vit_h return _build_sam( ΑΑΑΑΑΑΑΑΑΑΑ File "C:\ProgramData\miniconda3\Lib\site-packages\segment_anything\build_sam.py", line 104, in _build_sam with open(checkpoint, "rb") as f: ΑΑΑΑΑΑΑΑΑΑΑΑ ΑΑΑΑΑΑΑΑΑ FileNotFoundError: [Errno 2] No such file or directory: 'data/ckpt/sam_vit_h_4b8939.pth' [process exited with code 1 (0x00000001)]
The standard models are not made for inpainting! With SD 1.5 models you can do an on-the-fly merge by bascially subtracting the 1.5 inpainting checkpoint from the regular base model and adding this to the checkpoint of your liking. Then you will get way better results when inpainting stuff. Probably this can be done with SDXL too but since I still use SD 1.5 checkpoints most of the time, I can't tell for sure.
Thanks for still explaining everything in detail (conda, git, etc.) even after you've explained everyting a dozen of times already. This really helps.
Glad it was helpful!
I see a new Nerdy Rodent video I click instantly, Never failed me. Thank you!
You got the best vids man
I thought fooocus was good, but this thing is next level!
Brushnet or Invoke?
That shining suit of armor is suss! Love your videos though. 👍
I like the way it does the reflections too!
Excellent work as always mate. One quick question though. Where is the default output "dir" after generating the images?
What a funny intro 😂
😉
Ok so I am going to ask the obvious question.
How well does it fix hands and feet?
It's stable diffusion so probably not very well at all.
@@JonnyCrackers ? You can *fix* hands and feet quite easily in SD with inpainting. People do it all the time...
hey can you do a video on ANYDOOR, please ?
I read their github and it seems such a cool tool to change inject location and stuff, but sadly the GRADIO DEMO only has inject transfer from one image to another.
Can you make a tutorial on how to use all the other tools that Anydoor provides ?
This is really good, thanks !
I wasn't able to get it running. I keep getting a ton of errors, like "OSError: Error no file named diffusion_pytorch_model.bin found in directory data/ckpt/segmentation_mask_brushnet_ckpt."
But that file ends in .safetensors not .bin
You can download the required files from their Google Drive
Careful with those puns, Sebastian Kamph will become jealous. 🙂
The results look great but the one time I tried Anaconda it didn't work and seemed to be more of a mess. Then the hassle with the models so I think I'll wait until it hits Forge, which shouldn't be too long. Cheers NR.
Anaconda is the way forward and has never let me down in over four years!
@@NerdyRodent That's cool, it may have been something my end like a misunderstanding on my part. I also realised after that it might have been Miniconda, not Anaconda that I was using.
What file explorer and image viewer are you using?
Your setup looks like a mix of Linux and Windows in a good way.
I’m using Caja with the default image viewer 😉
I wonder what is more powerful, the IOPaint (that I noticed through your video last month), or this BrushNet. Evaluating after having seen both of your videos, I would tend to rather use IOPaint. But I am sure you must have compared both, or at least have a much better insight?
There’s a brushnet inspired PowerPaint for IOPaint now - best of both worlds!
@@NerdyRodent maybe we can expect some video from you about how to use this "PowerPaint" feature you described? Also a comparison of different models and how you use them in IOPaint would be very interesting. I cannot find a lot of input on how IOPaint can be power-used...
Great tut
@NerdyRodent I completed every steps without any issue for installation, but in the end, the local url is unavailable, it fails to connect to it, any idea why ?
If you’ve got extra security set up, you may need to modify it to allow local connectivity
@@NerdyRodent What'd'ya mean by extra security ? I tried with temporarily disabled firewall, checked my browser settings and flushed my DNS, that changed nothing wether I use Chrome, firefox or Edge. Now maybe there is a security setting within Brrushnet that I'm not aware of, if that is the case please could you tell ?
If you’ve not got any extra security running, it could be that you’re using Microsoft Windows which will fail on basic routing tasks. If that’s your case, you’ll need to use localhost as the address instead!
is the python method give higer quality/resolution results?
Vs which non-python method?
@@NerdyRodent or i should say locally vs website/huggingface
It’s the same running the program on your computer or one someone else’s 😀
@@NerdyRodent aright just making sure thanks, man !
Automatic1111 isn't deprecated?
Does kinda seem deprecated at this point, yes 🫤
I'm using Forge which is just another implementation of it.
I think your differential diffusion workflow doesn't use its full potential for its blending capabilities. Ideally after segemting the bear you could expand the mask and blur the expanded area so you get a gradient arround the bear. (or even blur without expanding to give it a bit of bear to change) Depending on the blur parameters and the added padding, your results should be much better. The differential diffusion allows you to use non binary masks and let the gray level define the allowed changes afaik.
Yup, I’ve got a node in there for you to set the blur amount though I find the depth maps are often pretty good without extra blur!
Seems like the checkpoints in Google Drive are hosed!
Google drive still working here!
Thank you so much!
You're welcome!
@@NerdyRodent Could you help me with this if you can please?
I tried to find out what is the problem but nothing was found :(
I have all the data structure and checkpoints and have the "sam_vit_h_4b8939.pth" but I don't know why this happens when I run the app_brushnet.py.
Traceback (most recent call last):
File "D:\AI-Programs\BrushNet\examples\brushnet\app_brushnet.py", line 13, in
mobile_sam = sam_model_registry['vit_h'] (checkpoint='data/ckpt/sam_vit_h_4b8939.pth').to("cuda")
File "C:\ProgramData\miniconda3\Lib\site-packages\segment_anything\build_sam.py", line 15, in build_sam_vit_h return _build_sam(
ΑΑΑΑΑΑΑΑΑΑΑ
File "C:\ProgramData\miniconda3\Lib\site-packages\segment_anything\build_sam.py", line 104, in _build_sam with open(checkpoint, "rb") as f:
ΑΑΑΑΑΑΑΑΑΑΑΑ ΑΑΑΑΑΑΑΑΑ
FileNotFoundError: [Errno 2] No such file or directory: 'data/ckpt/sam_vit_h_4b8939.pth'
[process exited with code 1 (0x00000001)]
Looks like you forgot to download that model!
Are all the output images low resolution?
The standard models are not made for inpainting! With SD 1.5 models you can do an on-the-fly merge by bascially subtracting the 1.5 inpainting checkpoint from the regular base model and adding this to the checkpoint of your liking. Then you will get way better results when inpainting stuff. Probably this can be done with SDXL too but since I still use SD 1.5 checkpoints most of the time, I can't tell for sure.
Hmm. Interesting. Do you have a vid link or tutorial you can point me to? :)
👋
I noticed that the quality of the generated images are really low quality...
Dont see any difference between this and normal inpaint models.