Great video, please keep producing great content. I have switched over to primarily using Krita instead of comfy b/c of your videos. PSA: For anyone struggling with Regions, it took me some time to realize that Regions don't fully work with 1.5. You can use the "generate active region only" in 1.5 using the alpha button (next to the generate button), but when i try and follow the instructions in this video with a 1.5 model (I tried several) and generate all regions at the same time, it completely ignores the regions. When i switch to a SDXL model, it works perfectly. There may be a workaround for 1.5, but i haven't figured it out yet, but SDXL seems to work fast enough in Krita that it's probably not necessary
Thanks! I was still using 1.5 almost exclusively up until a couple of months ago. Pony Diffusion models are my go to now, so I haven't tested the 1.5 models with Krita's newer features.
I'll probably just roll the random dice most of the time because I'm lazy. But I'll have to try integrating it with Krita someday. Thanks for the nice vid.
@@IntelligentImage I learned that when removing object from pictures. You have to try and change the prompt and change the amount until it finally gets rid of the object and creates a nice new blend.
Great tutorial! Thats really amazing how to separate things by layer! I tried exactly like you said and didnt work. Redo some times and still not. Wondering if this is broken in the last plugin version :(
Btw i realised it works in parts in SDXL, (but the characters are not created inside the layer like yours) they appear in the final image, a little bit off the painted area. Weird!
Ah is always exiting to see your videos pop up! do you post your works somewhere or have a discord to share stuff, etc? definitely trying this myself is so awesome!!!!
Thanks! I think I'll start posting stuff on X (Twitter) for now. I made an account and haven't done much with it. x.com/intellimageai?t=5XAAddlTYLPWefAm4ef3WA&s=09 I need to look into what Discord actually is. I've looked at it a few times and couldn't figure it out. Maybe I'm too old.
@@IntelligentImage , please let me know which model you are using on this example. I'm using Pony Diffusion V6 XL and I don't get fullbody characters as you do, I only get a very close-up of faces.
@@Lunarsong.Don't use the base pony. It's busted in many ways. Use autismMix instead. It's pony based model and lora compatible while being more predictable. Also pony based model have issues with inpaint and controlnet. For a beter experience try to use some other proper SDXL model that actually don't broke controls when you don't make anatomy or nsfw work. (wildcardXL, Juggernaut,AAMXL to name a few)
I'm using Pony Pencil XL with a lot of LoRAs. My next video I'm going to go more in depth into setting up Pony Diffusion in Krita. civitai.com/models/432249/ponypencil-xl
I love how you demonstrate what you're describing as you're doing it. One thing that I'm curious about is if your methodology will allow for incorporation of OpenPose wireframe models to get the posing right?
I feel like I underutilize control nets. I haven't actually tried them with regions. I would expect it to work. I would try and make sure the control net doesn't overlap different regions.
This looks great! Maybe I'll re-install Krita and the plugin huh. I have ComfyUI and models already installed and set up. IIRC, Krita plugin reinstalls an older ComfyUI version, but is it possible to link models from the original ComfyUI folder? I don't know if my question is clear..?
You want to share models between two Comfy UI installations right? You should be able to do it in the way described here: github.com/Acly/krita-ai-diffusion/wiki/common-issues#how-to-share-models-from-another-folder-with-krita-ai-diffusion-plugin
Fan-tas-tic! I’ve been holding back moving to the latest version of Krita AI, worried it’s going to be another learning curve. You’re helping reduce those fears. Btw, what version of Krita AI are you using? Also, have you updated to the newest version of Krita itself?.. Any issues? Thank you!
Thanks! Great to hear! I have been using the 1.22 version of the plugin. It looks like there is already an update I haven't installed yet. I'm pretty sure I have the latest version of Krita installed. The only issues I have had are that the newer versions of the plugin require additional models and custom nodes for Comfy UI. If you are not using the managed server, you will have to set those up manually.
Have you tested Flux dev with Krita AI? It seems to me that Flux doesn't follow regions as well as SDXL, but I might be wrong. I need to test this more
A question, can the loras only be used through a connection or can I download them and use them locally? I have a gpu gtx 1080 ti, do you think it can run on my PC?
also a video where you can (if possible) fill. for example in a group photo subject a is standing infront of subject b, it will fill the missing limbs of subject b
I won't be going over these things specially, but the techniques and concepts I will be showing in my upcoming video could be used to achieve both of these things.
Do they show up where you can apply them in the settings? If there is only one Lora missing and it is one you just installed, you may need to restart Krita to get it to show up.
I'm really not sure then. It should work the same way in the text prompt node in ComfyUI. You could see if it works there and at least determine if it's a Krita issue.
Some models have some character baked in. For example AnimagineXL have the hololive Vtuber but not PonyXL. When a character is absent, Lora can be used on the mentionned checkpoint. For background it's good to have that as separate layer. For consistent style, a LORA. For likeness overall, either reference/style/composition controlnet.
adressing PONY XL issues : ITS NOT KRITA FAULT / it's purely pony that is pure jank. actually Pony based model is so broken in it's core that it bring a lot of problems so here is the list : 1 - broken lora compatibility. 2 - WAY less controlnet effects and UNION controlnet is so random. 3 - Fooocus inpaint patch is now useless and it's prety mandatory to avoid having an inpainting model. 4 - by extension to 1) Live mode won't works since it rely on a Hyper LORA. 5 - by extension to 1) IPadapter is very unpredictable. --- this is problematic since i found that PONY train on new character data much easier which is great for originals characters. that make it my favorite model to train and use but it doesn't adress such glaring flaws.
I would try composing images with non Pony models and then doing image to image with a Pony model to get the Pony style to get around most of these problems.
Man you have opened my creativity for my work with ai so much. Your Knowledge is greatly appreciated. Can't thank you enough, so thank you.
Thanks! That's great to hear! I'm open to suggestions about future content also 😀
Great video, please keep producing great content. I have switched over to primarily using Krita instead of comfy b/c of your videos.
PSA: For anyone struggling with Regions, it took me some time to realize that Regions don't fully work with 1.5. You can use the "generate active region only" in 1.5 using the alpha button (next to the generate button), but when i try and follow the instructions in this video with a 1.5 model (I tried several) and generate all regions at the same time, it completely ignores the regions. When i switch to a SDXL model, it works perfectly. There may be a workaround for 1.5, but i haven't figured it out yet, but SDXL seems to work fast enough in Krita that it's probably not necessary
PSA Update: The Regions seem to work MUCH better in 1.5 if you DON'T include anything in the "common text prompt" field or in the background field.
Thanks! I was still using 1.5 almost exclusively up until a couple of months ago. Pony Diffusion models are my go to now, so I haven't tested the 1.5 models with Krita's newer features.
Aaaah i was waiting for someone to cover it. Let's watch that
I'll probably just roll the random dice most of the time because I'm lazy. But I'll have to try integrating it with Krita someday. Thanks for the nice vid.
Thanks for sharing all this information. You're definitely the master of using the Krita plugin to generate truly great art. 🙂
Thanks! It takes a lot of tries to get things to turn out right though 😅
@@IntelligentImage I learned that when removing object from pictures. You have to try and change the prompt and change the amount until it finally gets rid of the object and creates a nice new blend.
Great work!
Thanks!
PONY!!!
More on that next time!
Genius
Thanks for your tutorials,
Glad you like them!
This technique is amazing!
Awesome channel
Thanks!
You go that hollywood actor or model face. Beside that, thank you for the knowledge.
Thanks 😅
Great tutorial! Thats really amazing how to separate things by layer! I tried exactly like you said and didnt work. Redo some times and still not. Wondering if this is broken in the last plugin version :(
Btw i realised it works in parts in SDXL, (but the characters are not created inside the layer like yours) they appear in the final image, a little bit off the painted area. Weird!
Thanks! I've found the regions can be a bit buggy at times. A new version of the plugin was released today. It may be worth trying the upgrade.
Ah is always exiting to see your videos pop up! do you post your works somewhere or have a discord to share stuff, etc? definitely trying this myself is so awesome!!!!
Thanks! I think I'll start posting stuff on X (Twitter) for now. I made an account and haven't done much with it. x.com/intellimageai?t=5XAAddlTYLPWefAm4ef3WA&s=09
I need to look into what Discord actually is. I've looked at it a few times and couldn't figure it out. Maybe I'm too old.
@@IntelligentImage😂😂😂
@@IntelligentImage , please let me know which model you are using on this example. I'm using Pony Diffusion V6 XL and I don't get fullbody characters as you do, I only get a very close-up of faces.
@@Lunarsong.Don't use the base pony. It's busted in many ways. Use autismMix instead. It's pony based model and lora compatible while being more predictable.
Also pony based model have issues with inpaint and controlnet.
For a beter experience try to use some other proper SDXL model that actually don't broke controls when you don't make anatomy or nsfw work. (wildcardXL, Juggernaut,AAMXL to name a few)
I'm using Pony Pencil XL with a lot of LoRAs. My next video I'm going to go more in depth into setting up Pony Diffusion in Krita. civitai.com/models/432249/ponypencil-xl
I love how you demonstrate what you're describing as you're doing it. One thing that I'm curious about is if your methodology will allow for incorporation of OpenPose wireframe models to get the posing right?
I feel like I underutilize control nets. I haven't actually tried them with regions. I would expect it to work. I would try and make sure the control net doesn't overlap different regions.
THANK YOU
Thats a possible to add filter for making a pixel art with this?
You could try using a pixel art style checkpoint such as this one: civitai.com/models/277680/pixel-art-diffusion-xl
This looks great! Maybe I'll re-install Krita and the plugin huh. I have ComfyUI and models already installed and set up. IIRC, Krita plugin reinstalls an older ComfyUI version, but is it possible to link models from the original ComfyUI folder? I don't know if my question is clear..?
You want to share models between two Comfy UI installations right? You should be able to do it in the way described here: github.com/Acly/krita-ai-diffusion/wiki/common-issues#how-to-share-models-from-another-folder-with-krita-ai-diffusion-plugin
Fan-tas-tic! I’ve been holding back moving to the latest version of Krita AI, worried it’s going to be another learning curve. You’re helping reduce those fears.
Btw, what version of Krita AI are you using?
Also, have you updated to the newest version of Krita itself?.. Any issues? Thank you!
Thanks! Great to hear! I have been using the 1.22 version of the plugin. It looks like there is already an update I haven't installed yet. I'm pretty sure I have the latest version of Krita installed. The only issues I have had are that the newer versions of the plugin require additional models and custom nodes for Comfy UI. If you are not using the managed server, you will have to set those up manually.
05:36 in Comfyui that happens when we try to use a sd15 model in a XL workflow or the other way around
Interesting. I suspected it may be a general problem of sizes getting misaligned somewhere in the workflow.
Have you tested Flux dev with Krita AI? It seems to me that Flux doesn't follow regions as well as SDXL, but I might be wrong. I need to test this more
I haven't tried flux yet. I wouldn't be surprised if it didn't respond to regions in the same way.
A question, can the loras only be used through a connection or can I download them and use them locally?
I have a gpu gtx 1080 ti, do you think it can run on my PC?
You would have to download them and use them locally. It should run on your PC, but I'm not sure about the speeds you will get.
Hey, I have a question, but I don't know where to contact you to ask?
You can DM me on x/twitter. There is also an email link in my channel bio. x.com/intellimageai?t=Z8azcVj42vbhaM4m9zz5PA&s=09
thank you so much, could you make a video where you turn an image into a style? example game screenshot into a specific style, with same pose?
also a video where you can (if possible) fill. for example in a group photo subject a is standing infront of subject b, it will fill the missing limbs of subject b
I won't be going over these things specially, but the techniques and concepts I will be showing in my upcoming video could be used to achieve both of these things.
lora doesnt seem to showup when i type ?
Do they show up where you can apply them in the settings? If there is only one Lora missing and it is one you just installed, you may need to restart Krita to get it to show up.
@@IntelligentImage yes they showup in settings but not when i type
Where are you trying to type it? It only works in the prompt box of the docker itself, not in the style prompts in the settings.
@@IntelligentImage ya in the docker itself just as in video
I'm really not sure then. It should work the same way in the text prompt node in ComfyUI. You could see if it works there and at least determine if it's a Krita issue.
I've not tried it out still. Is there any way to create likeness images. Like for specific character or a person?
Some models have some character baked in. For example AnimagineXL have the hololive Vtuber but not PonyXL.
When a character is absent, Lora can be used on the mentionned checkpoint.
For background it's good to have that as separate layer.
For consistent style, a LORA. For likeness overall, either reference/style/composition controlnet.
You can also try the composition and face control nets to reference a specific image.
@@IntelligentImage I till investigate, thank you, will let you know about my results
Do you have a discord by any chance?
I'm working on it 😁
adressing PONY XL issues : ITS NOT KRITA FAULT / it's purely pony that is pure jank.
actually Pony based model is so broken in it's core that it bring a lot of problems so here is the list :
1 - broken lora compatibility.
2 - WAY less controlnet effects and UNION controlnet is so random.
3 - Fooocus inpaint patch is now useless and it's prety mandatory to avoid having an inpainting model.
4 - by extension to 1) Live mode won't works since it rely on a Hyper LORA.
5 - by extension to 1) IPadapter is very unpredictable.
---
this is problematic since i found that PONY train on new character data much easier which is great for originals characters. that make it my favorite model to train and use but it doesn't adress such glaring flaws.
I would try composing images with non Pony models and then doing image to image with a Pony model to get the Pony style to get around most of these problems.
the upscale face is a bit overthinking. is easier to upscale the whole image up... inpaint... than ... upscale down. , no need to realign