PhotoMaker - better than IPAdapter?
HTML-код
- Опубликовано: 13 май 2024
- Photomaker - create photos, paintings or avatars for anyone in any style within seconds! Quicker and easier than training a LoRA... but how good it is? Can it beat IPAdapter? You decide!
Works great locally on Linux, MS Windows and Mac - or via a Huggingface space for the non-techies / those without hardware!
Unlike InstantID (which can’t be used for commercial purposes = no pay-for installers, monetised RUclips videos, etc), Photomaker isn’t dependent on insightface! Yay!
Want to support the channel? Well now you can! :)
/ nerdyrodent
== Links! ==
Project - photo-maker.github.io/
Original Repo - github.com/TencentARC/PhotoMaker
MS Windows Repo - github.com/bmaltais/PhotoMaker
Huggingface - huggingface.co/TencentARC/Pho...
SDXL Model - huggingface.co/SG161222/RealV...
LoRA - civitai.com/models/124347?mod...
ComfyUI - github.com/ZHO-ZHO-ZHO/ComfyU...
== More Stable Diffusion Stuff! ==
* Faster Stable Diffusions with the LCM LoRA - youtu.be/
* ComfyUI Workflow Creation Essentials For Beginners - • ComfyUI Workflow Creat...
* How do I create an animated SD avatar? - • Create your own animat...
* Installing Anaconda for MS Windows Beginners - • Anaconda - Python Inst...
* Add anything to your AI art in seconds - • 3 Amazing and Fun Upda...
* Video-to-Video AI using AnimateDiff - • How To Use AnimateDiff...
* One image Gets You a Consistent Character in ANY pose - • Reposer = Consistent S...
Contents
0:00 Photomaker install and testing
7:57 Photomaker in ComfyUI 
Наука
"A picture paints a thousand words, or is it the other way around now?" - Love this quote.
I've just been waiting for this video for three days!!! :D :D you are the besttt
Great little tutorial, thanks!
Really crazy how you actually got the comfyui nodes to work. No one else other than you and the repository creator seem to be successful judging by the github issues.
hehe one of these issues was posted by me, i was lucky to solve my problem..
but i was not so happy with results and flexibility comparing to ipadapter
Obviously only people with issues will post issues 😉 Try the “install from GitHub url” option in manager - especially if you didn’t use a standard ComfyUI install
A rodent that is nerdy enough can do anything!
Another great video! Would love to see a video on the new dreamoving animation tool.
Good sir, you are a life saver.
can this be used for objects as well or just faces? great video btw. very fresh
Great video!! Where did you get the Lora workflow from? I can’t find it. I only see the non Lora workflow. Also, how did you generate multiple images? Thanks in advance for your time to answer my questions!
The workflows are just under the install text 😀
hello is it possible to use this with a1111?
Thanks. Is there any way to copy the facial expression to a specific photo?
I have two GPU's how do make sure comfyui is using the most powerful of the two nvidea cards?
could this be added to the reposer workflow? Thanks again for all you do!
For sure, though with some slightly different nodes 😉
@@NerdyRodent a bigger brain than mine is needed for that me thinks. a job for Mr Rodent perhaps? ✌️🙏🙏✌️💐
That was super solid, thank you @nerdyrodent! 🤠
I keep getting errors when going LoraLoader. Peft backend is required for 'set_adapters()'. I upgrade the pip and setuped the peft on disk C:.
Thanks Nerdy, I enjoyed it.
don't know why but not able to get any similarity to input image
does this work for SD1.5?
Am getting an IMPORT FAILED error even after pip installing. What could be the problem?
After installing the custom nodes and everything else, I still get the error that
When loading the graph, the following node types were not found:
BaseModel_Loader_local
PhotoMakerAdapter_Loader_local
PhotoMaker_Generation
Ref_Image_Preprocessing
What am I missing here?
same here
You’ll need to install the photomaker custom node (git clone, etc) then restart ComfyUI
photomaker_generation already replace by NEW photomaker generation -> remove it and go to add node / photomaker , you will find the new node there
@@NerdyRodent Same here, I also have the the following directories setup :
custom_nodes->ComfyUI-PhotoMaker (with all the correct contents inside)
I git cloned, I installed requirements.txt and get the same errors.
Also restarted and tried reinstalling requirements.
(edit)
So what fixed it for me was deleting the ComfyUI-PhotoMaker Folder in the custom-nodes then on comfy ui manager I installed it from the url on the comfy ui manager itself
@@jeandresadie3396 They updated the workflows. Use the V2.5 Json local file
How do you get these sliders on the nodes?
It looks interesting, I will give it a try, but I've been very impressed with faceid with ipadaptor. I'm curious why you think Linux is the best OS for AI, I've used different distros for years, from the early days building Gentoo and the first release of Ubuntu, to todays Fedora desktop. Nvidia consumer GPUs have well been proven to be the best for AI related tasks in comparison to AMD and Intel offerings, but I've always found Nvidia performance to be worse on various Liniux distros than on Windows.
The main reason Linux is better AI than Windows is that most development is done on Linux first and then ported to Windows which is less optimized and can even introduce bugs.
The second reason is that if you use your GPU for both AI and displaying your GUI, the AI processing needs to be interrupted to redraw the GUI. The way Windows does that is less performant than Linux wich explains the faster rendering times you get on Linux.
@@fooltopia That's not what I've experienced, on the same system, switching between Windows 11 Pro and Fedora Desktop 39, I have consistently better performance on the Windows desktop. I'm aware that a lot of development is on Linux based systems, as is a lot of software for a lot of different fields, however you compile software for a target platform and that doesn't mean it will suffer from degraded performance or anything else. If I compile a program for a target using Windows or Linux makes no difference. What will affect performance is individual configurations of hardware.
@@weirdscixThat's interesting. Do you have an NVIDIA GPU? Are you using that for your display as well, or do you have a secondary GPU? I read somewhere that the A-Series GPUs use a different driver that handles Windows better.
Of course there are always exception, but on average, Linux seems to perform better than Windows when it come to generative AI. But since I'm using Windows, I would love to learn more about your setup so I can improve my performance as well.
@@fooltopia I have one machine with a 3090 which is my main machine and the one I play around with AI on, and the other has dual 3080 Ti which I mainly use for Unreal Engine projects which are non NVlinked. I haven't tested much AI related projects on it
its taking more than 30 minutes to load. i am unable to generate images since yesterday. it just get stuck in loading
how much VRAM of gpu do you have? I have a 4070 with 12gb and it shows a cuda allocation error :/
This is really interesting. I had it almost working, but when it got to the adapter loader local I got this error, wondering if anyone has an idea of how to get around this:
Error occurred when executing PhotoMakerAdapter_Loader_local:
'StableDiffusionXLPipeline' object has no attribute 'load_photomaker_adapter'
same issue here
same
8:58
I finally got it working after using another workflow. The resemblance is way worse than Ipadapter, though it’s more flexible in different styles. I guess the best solution for resembling is faceid portrait + faceidplus v2, but it’s not flexible for styles.
This one is also good to go for commercial use, unlike the the ones using insightface 😉
@NerdyRodent There is another project called " InstantID " , it is been confusing between IPadapter , PhotoMaker & InstantID . Do you know what's the difference between all of these ?
InstantID is for non commercial use only due to insightface
When AD will work with this, it’s gonna get nerdy ;)
@Nerdy Rodent, excellent video! however i cannot get it to run on windows in comfyUI, it does not accept the SDXL path. Is it possible to change the nodes to a selection menu, like most comfy nodes? the warning is "Cannot load model SDXL/RealVisXL_V3.0: model is not cached locally and an error occured while trying to fetch metadata from the Hub. Please check out the root cause in the stacktrace above."
You’ll need to download that Sdxl checkpoint to use it. You can also use any Sdxl checkpoint you already have! 😃
@@NerdyRodent Thanks for the reply but that checkpoint is in the folder i downloaded the exact same one, it does not recognize it, i assume because the base directory is not set to "checkpoints" because the path is linux specific?
As you’re opting to download from the hub, just keep the repo path as is, without changing it. If you’re doing it like in the video, click the node to select your checkpoint. Check out my patreon for extra workflows, or to dm me!
Great video. Was hoping for a ComdyUI implementation. But do you know why it is using a custom sampler node? And is it compatible with the default nodes. Like could you combine it with IPAdapter to see if they could work together?
As it’s only been out for a few days, everything is still pretty new. While zho’s nodes make for a small and easy workflow, unfortunately they’re not really compatible with anything else 😕
I'm trying to run this workflow, but no sucess. It seems IP Adapter released two new models: FaceID Portrait and FaceID SDXL. Would love to see a comparison
They use insightface so far as I know, so no commercial use allowed 🫤
Anyone else getting this error? 'PEFT backend is required for 'set_adapters()'
No known solve from what I can find...
i have it too any solution?
I keep getting errors when loading lora. My video memory is 12G. Is there any reason?
Thanks for this, but I keep getting "PEFT backend is required " when it gets to the PhotoMaker node.
I have the same issue - online solutions all appear to say just update PEFT and Transformers - done that and still not working. Anyone who has sorted this, please post the solution to help those of us who are struggling to get beyond it
@@dcpuzzles2990 Did you ever find a solution?
same issue
Painfully backwards node
Does PhotoMaker work with DirectML and AMD RX580 on Windows ? The version that I installed seems to run on CPU only and it's completely useless because It takes forever to generate an image.
For AMD you’re generally better off using Linux + rocm PyTorch
@@NerdyRodent Do you know which version of Linux or which distribution works with the old RX580 ?
@@lucian6172 Ubuntu is the standard, but I know nothing about amd hardware as it’s not really used in AI 😞
Thank you! Got it work with comfyui.
After comparing side-by-side with FaceID and Plus Face model, it's clear that the ipadapter stands in a league of its own. However, as you mentioned, it's not intended for commercial use.
Plus on top of that, it’s harder to change the style. Try doing a comic art or papercut style, for example…
GPT
Yes, it is indeed difficult to change the style with ipadapter. E.g. Disney style😅@@NerdyRodent
Grunacho, did you use the SDXL faceid ip-adapter or SD1.5? Because everyone's saying that the SD1.5 is better.
@@MyAmazingUsername At the moment im using sdxl-faceid with combination of faceplus sdxl model.Sdxl has not yet v2. I think the quality is higher & you can use it with sdxl turbo. Just my opinion.
@@Grunacho there’s a new faceid v2 for SDXL. Haven’t tried it yet. I’m struggling get good skin details in SDXL, epicphotogasm is kind unmatched. I wonder if a workflow with Hephaistos and epicphotogasm on a second pass would help.
It takes 8 minutes to create one image in comfyui for me using photomaker 😞
This looks like the evolution of LoRas
With in seconds?
Anybody can help me? Where can I find these nodes?
Ref_Image_Preprocessing
Prompt_Styler
NEW_PhotoMaker_Generation
PhotoMakerAdapter_Loader_local
BaseModel_Loader_local
These are missing, I use the manager to load the missing ones, but it's not working. So if somebody know where to search them pls share the information with me.
You can install them as shown in the video, or use install from GitHub
@@NerdyRodent Sorry, but I don't see in the video which part demonstrates where I can find, for example, this: Ref_Image_Preprocessing. Can you provide a link for that pls?
For ComfyUI, see 7:57 onwards!
@@NerdyRodent There was an issue; the 'pip install -r requirements.txt' did not complete properly. It got stuck at the end because Rust was not installed. It's good now.
thanks for the tuto but still have issue with red node nothing works i will wait for a simple version
I’ve uploaded an advanced workflow version on patreon
In what front ends does this run?
Native app is gradio, or you can also run it via Comfy
Good , Thanks, but how much vram on video card do i need for this and much motherboard ram ?
Is pretty much like Faceswap Lab on SD 1.5
Faceswap lab cannot be used for commercial purposes due to insight face
Instant ID for Control Net is the best way to turn photos into AI images so far IMO...it was added in the lastest CN update.
Just a shame InstandID isn’t for commercial use
i dont know if this is the gold rush or the race to nuclear power but i am in it.
Character consistency without a lora will be hot for some time I believe.
@@blacksage81 search goes on and on , probably we will run out of disk space in our pcs , keep trying different methods .lol
upvote for miniconda.
End screen was creepy 😥
I'm just here for the gaming vidz 🤪
😉
A1111?
Soon (tm) 😆
looking for a1111
I believe Photomaker is ipadapter :P Just once again another model haha.
Too bad! Installation always gives me some kind of RUST error. :/
Be careful what Python you use. Make sure to point Windows to the embedded Python
Is it A1111 compatible?
If not, it's a pass for me.
Those kind of things are supported much sooner (and better) in SDNext
Should be supported in A1111 soon…
awesome, is tis censored or uncensored?
It’s completely uncensored
Before I watch and get my hopes up. Can you do this on a Mac offline for free?
Unfortunately, you will need an Internet connection in order to be able to download the models the first time ☹️
"Donald Musk" is the dad from Problem Child.
Actually results are worse than IP adapter
Very slight resemblance. Their claim did not stand to the truth. Or stability isn't there yet.