This One Simple Plugin Adds Realtime AI Assistance to Krita
HTML-код
- Опубликовано: 13 май 2024
- If you're more used to using digital art programs like Krita, then this Live AI add-on may be just the thing for you. Choose how much or how little your image will change, draw as usual and see any changes happen instantly! It kinda adds a new dimension to scribbling, and with Krita as the interface to ComfyUI you don't see the noodles. Krita AI Diffusion really is a lot of fun, with the new dimension of realtime AI adding to an already amazing digital art program.
Best of all, this uses FREE software meaning you won't have to pay a thing. No subscription, just download, install and run in moments just by following this video tutorial!
Makes live painting via ComfyUI a doddle 😀
ComfyUI Workflows & Resources:
github.com/nerdyrodent/AVeryC...
Plugin download:
github.com/Acly/krita-ai-diff...
*As with most AI, a GPU with a minimum of 6Gb VRAM is required.
== More Stable Diffusion Stuff! ==
* Faster Stable Diffusions with the LCM LoRA - • LCM LoRA = Speedy Stab...
* How do I create an animated SD avatar? - • Create your own animat...
* Installing Anaconda for MS Windows Beginners - • Anaconda - Python Inst...
* Add anything to your AI art in seconds - • 3 Amazing and Fun Upda...
* Video-to-Video AI using AnimateDiff - • How To Use AnimateDiff...
* One image Gets You a Consistent Character in ANY pose - • Reposer = Consistent S...
Want to support the channel?
/ nerdyrodent
Chapters
0:00 Introduction to Live drawing with Krita AI
1:24 Installing the Krita AI Plugin
3:45 Enabling the Krita AI Plugin
4:20 Krita AI Plugin Managed Install
5:15 Krita AI for Existing ComfyUI Installations
7:12 Troubleshooting an existing ComfyUI install
8:24 Krita AI Plugin Options
9:16 Live AI Examples
Want to support the channel?
/ nerdyrodent
Thanks for watching! 
Наука
Krita is an amazing app by itself, with this it's out of this world darn!
Ikr!
@@NerdyRodent Can u use with forgeui installed in the background
Thank you my Nerdy Master for your time and be my guide!
I am already playing with this for the past week and this is a game changer for story board artists, concept designers and film pre-production! This will make existing workflows way way much more productive. Even film makers can now flesh out their story ideas quickly.
Thanks Nerdy!! Always a please to watch!! 😊
Glad you enjoyed!
This is marvelous ! Thank you very much again for keeping us up to date !
And thank you for enjoying the things!
Wow... this is a really amazing AI feature. To be able to upscale the art from literally line drawings to high resolution art in real time... it's really amazing. Thanks for showing this.
The best complete tutorial on this subject I have found . :O)
CLASS BRO. TNX TO THE SD TEAM AND YOU. ❤
This looks super fun, cant wait to set it up. Thanks !
I wish to see one day these workflows, combined with reposer and colored segmemt, can help people make their owm cartoon by more accurately generating images of consistent characters/outfit in various poses with higher compositional control. Great video!
Thanks Nerdy; plugin installed! 👍
Warning: it’s VERY addictive!
Can you help me?
I discovered that if you apply a noise filter layer onto the drawing, the results will be fantastic! Using dry textured brushes also gives much better results. Just to add on, you can create a new image layer, fill it with white, then add a noise filter. Use the Opacity adjustment to control the amount of noise on the sketch and watch the wild results appear. Changing the models/checkpoint types also gives a wide variety of results.
Great tip!
Could you show some images?
@@HestoySeghuro I'll try to explain the steps as clearly as I can. Create a new image 1024x1024. Create a new paint layer by clicking on the + sign to create "Paint Layer 02". Press Shift+Backspace Key to fill the layer with a color, any will do. Right mouse click on the paint layer 2 and select Add - Add Filter Mask - Choose the last option "Other" drop down triangle and choose Random Noise. Bring it to level 99. That's it you created the noise layer. Switch to live painting generation, slide the strength to 45 and above. In the layer opacity option reduce it down to 15%. Type your prompt, choose your brush and start painting away on Paint Layer 01. Keep adjusting the opacity of the noise layer to see different results. You may also adjust the brightness and contrast to see more variations.
Love from Sweden❤ Keep on with these amazing Videos😊
Wow, many thanks! Glad you like ‘em 😀
Kinda sad this didn't grab a lot of attention. It's been available for a long time now but some extensions died because they lacked contributors. The live factor is fun
Yup, live is indeed a lot of fun!
drives me insane right@@VioFax
I tried it today and it seems kind of broken
What extension? I think is cause Adobe peer pressure.
It's too damn complicated is why. I got the main program installed, got the AI plugin up and running and that installed the Comfy thing, but there's no real instructions how to use it, hence I'm here, and this vid veers off into comfy ui, which by itself is some codey thing for nerds
Excellent choice of thumbnail/1st example. You clearly know what you are doing XD
This tool seems amazing. Gotta try it!
Thanks. It’s amazing how even expert artists, such as myself, can experience AI!
I tried earlier this year, will check again. Can’t wait to hear your take on StableDiffusion video.😅
If svd is ever available for commercial use, then I’m sure we’ll see a lot of it! Personally, I think it’s pretty cool. Like some strange auto depth masking movements
@@NerdyRodent 2024 is going to be wild
I tried this week, and blurgh! I have 3 different ways of running Stable Diffusion and this is by far the messiest and most complex. It offers more but it's just such a headache I'm not sure it's worth the bother? I'm not a paid artiste, I just wanna make pictures an edit em.
You never fail to amaze!
Are there any other live drawing-AI tools like this? This is the first time I've seen that in action and it's amazing. Thank you for this tutorial.
Just mindblowing! I'm so excited with this AI in design! As well with KDE, meaning that anybody shouldn't pay for Adobe anymore
Very cool krita comfyui setup. I had been using a version by someone else but it doesn't have this cool real time stuff or built in ipadapters. Gonna have to check this one out. Thanks a bunch nerdy!
Yeah, this is the best one I’ve tried so far!
I was about to ask for a video on this.
And thus, is is provided!
This is awesome, stumbled across your channel as we've started to explore ComfyUI over 1111. Krista, Why didn't I know about this earlier? Desperate to move away from Photoshop and Gimp wasn't quite cutting it. If nothing else prompting SD in this way really helps you get a feel for the way the model works, I absolutely love watching the live picture update as I sketch, but also as I update my prompt, add commas etc etc. This is great, thanks for highlighting it.
Yup, it’s great to see your changes happen as you make them!
For those who have trouble adding a open pose character to a layer, make sure the layer is a vector layer.
Insane!
This is great !! thank you Nerdy Rodent:)) do you know if the images are saved anywhere, the in progress images it generates ?
Maaaaaaaaaaaaaaaan, you're the best. Come doing the nerd shit you do, I'm your fan!
Awaken your inner nerd! 🤓
THANK YOU, realy good ! Is it working with stability matrix? Thank for you sharing :)
Thank you so much.
You're welcome!
Hi there and thank u for this amazing video. It seems impossible to use the AI plugin. I went through all the steps and the lengthy download but what I get is an error in the Server configuration which reads "Invalid location: directory is not empty, but no previous installation was found". In the "Core Components" I see only Python as installed but not the other 3. Is there something I missed??
thank you for showing us and how do i get back a previous image?? is there anywhere we can view all the images generated? cos I think i liked one i did four images back but how do i re generate it?
The best part is I can just bash together random images together, just put in some images I find on google (or generate it) and comp together a scene, and the live painting does the magic of making it all look seamless!
okay this is ... alright, but ... can we change the prompt workflow? I'd like to add in my character as a ip adapter and use a different model etc... this feels difficult to use well, for drawing from scratch?
For anyone without the hardware power to run locally there's a hugging face ui space with this real time tech
I have it running on 4GB VRAM (Linux) if you don't mind waiting 20 seconds instead of 2 seconds. People are just too damn impaitent these days. 😁 It's much better doing inpainting in Krita, rather than ComfyUI's mask editor.
Nice!
Hey Nerdy Rodent, thanks for your tutorials, could you please make one on training LoRAs with a 3060 12G (Linux) to be used with this Krita plug-in?
is it working with an AMD video card ? tks
i can get full img generation working, and outline selection generation, they work fine. but whenever I try the realtime drawing it just returns me the stick figure I drew in the first place, rather than a rendered image. I'm confused.
Thanks Mr Rodent. Finally i can use this properly (already had it installed) but i didn't know how to add the control layers etc.
Live diffusion ... where were we again a year ago? Insane progress. xD
Glad I could help!
nice explanation about this! thank you! do you know if it also works with the confy ui extension installed in automatic1111? or we need a separate confyui installation? it would be great to know for not tu fill to much the HDD!
So long as it works in exactly the same way as the normal ComfyUI, it should be fine I guess!
@@NerdyRodent I'll try! is an amazing add-on!
I would like to give this setup to a kid and see them go crazy.
Yup, almost any age can use Krita - just make sure to use a "safe" checkpoint :)
I'm a bit confused about the server configuration page. Under core components I see ComfyUI is installed but required models is not, does that rectify by the 10G ComfyUI install? Also under Workloads Stable Diffusion 1.5 is checked but not installed so does that install with the above ComfyUI install if it's checked? I do already have Stable Diffusion on my computer so I wonder if there is some redundancy or if it's a matter of the AI plugin needing it's own copy as there is no integration with a preexisting Stable Diffusion installation? Same with installing checkpoints, I have Realistic Vision and DreamShaper installed in my Stable Diffusion but not separately for the AI plugin and I also have ControlNet installed separately to my stable Diffusion.
you are a born voice actor, just phenomenal, can you clone your amazing voice for us all poor speakers to have fun?
i don't have graphic card. i install stable diffusion on kaggle notebook. if i add that address in Krita sever. does it work
support server stable diffusion or not if we don't have ComfyUI sir ?
Hello i have an issue. When i try to install ComfyUI i get an error "utf-8 codec cant decode byte 0xd0 in position 25: invalid continuation byte". Any thoughts on why this might be happening ?
I have big issues with the Zip fil in the folder 😢
nice
Unfortunately, the plugin never actually generate images for me. Regardless of whether it is live mode or generated, the finished image remains blank.
how we update the plugin?
This might be a stupid question but where is the prompt negative?? I know where the prompt negative for the preset is, under settings, but how can you get a per-image negative?
Cảm ơn video đã hướng dẫn tôi cặn kẽ, tôi đã có thể chạy được chương trình và bắt đầu vẽ, khá là vui vẻ.
This is brilliant!! :D
I was sad before because a couple of Krita plugins for Stable Diffusion I tried in the past were discontinued... And now thanks to you I learn this one is working and even updated to use the LCM-Lora! I'm so happy! You are the best, Nerdy Rodent!! Thank you very much!! :D
Glad I could help!
I can't get the generative image to show, everything is working but there is no generation image?
I installed this and got the plugin running, but every time I try to run a prompt, python quits just before it finishes. So disappointing. Any ideas for a fix? I'm running it on a Studio Mac.
Please help. Server execution error: insufficient funds- generation would cost 8 tokens. Remaining tokens: 3
I don't get that tiny little box and don't have access to live draw. what's going on?
hello! Im having problem runing the plug in using the local server it gives me this issue : invalid location: directory is not empty, but no previous installation was found and wont allow me to install anything :(
Try renaming your existing directory
How to add model
😵💫Awesome
I think I need a little help with this. I'm trying to do the exact steps you said in the video but for some reason my Comfyui just says "Queued" and nothing is happening.
I have no workflow loaded into comfyui because you never said to use one in the video. I also never selected any LCM models or anything because that wasn't said in the video either.
Does comfyui not need a workflow? Does it automatically choose a LCM/turbo/lightning model when you click live painting?
I guess I'm a little confused as to how this is working. Any help would be appreciated. =]
Yup, this is a plug-in for Krita with ComfyUI as the back end
@@NerdyRodent I totally figured it out!
My image was too big. So it must have been getting an error in the backend without me seeing in Krita. I scaled the image size down to something more manageable and it worked.
I was having some issues with it not working when I used the selection tool. I just reinstalled krita and used the Local Install so Krita manages everything and it's working flawlessly now.
In case anyone else has any issues.
Thanks for the video!
LCM has never been my "friend". What are your settings in Krita's Generator to get the highest speed & good question here? I keep getting garbage
I just use the defaults :)
After allowing Krita to install ComfyUI, is it possible to use ComfyUI independently of Krita and what would the steps be? Is it just a matter of configuring run_nvidia_gpu.bat properly?
I would imagine you could, yes
Are you going to cover stable video diffusion?
It is just a research only model at the moment, but once something is available commercially, I’m sure there will be lots of videos!
Do you know? Could this be done just in Comfy? That way I could set-up a camera, run the workflow through Touchdesigner which would mean I could draw the things on paper, instead of a computer.
Yup, there’s the canvas live paint node but obviously nowhere near as advanced as KritA
I can't get Krita to see where Comfyui is already installed. I've tried every path, it just can't see it. I'm about ready to uninstall Comfyui and let Krita install it.
great video but wondering if it can work on 4gb vram?
Probably but not as quickly, maybe if you stick to a 512x512 canvas.
how lauch comfy? which file should I run in folder ai_diffusion?
If you already have ComfyUI installed, simply start it as normal. You don’t need the web page open. If not, the app manages it for you.
Hi all there. I downloaded the plugin and put it in the resources directory as the video said, BUT it doesn't show up in the python manager so i cant procceed. Did anyone have the same problem and how is it fixed. Thanx. And something for Nerdy Rodent i subscribed to the channel a long time ago but today youtbe showed that i wasnt...just a heads up
I had the same issue, after moving everything in zip right under pykrita, it works.
I'm having the same issue, but I'm too stupid to understand the fix. I'm brand new to Krita after the YT algorithm showed me this video!
Nevermind, proof I'm an idiot. I just needed to run the proper Python Plugin installer.
@@SugarySnackMan what python installer...where is it
This is really good, I've been using krita extension in automatic1111 for 3 months and my workflow has improved but this one on comfyui is the next level. Do you know if I could use this extension not interfering with automatic1111?
Yes, this is totally independent of a1111
Dude, I don't know who you are, but you're definitely the coolest and most inventive process maker👏
Does this also work with pascal era gpus?
Some people run Comfy on the 980ti, so you should be good!
@@NerdyRodent Nice. Really appreciate your videos btw!
do I need 15G GPU? or works with any paid server?
I am using 4GB vram laptop and it works brother. the live option is a little slow but works.
this is crazy! can you also apply trained data, like what corridors crew did to apply a specific art style?
Sure, you can pick any model you like! Didn’t notice loras there though
Is there anyway I can train the AI to learn my art style? I don't want the AI to just pick images on the internet.
Yup! Dreambooth, textual inversion and LoRAs can all be used to fine tune any model. Check the links in the video description for more info!
krita says ipadaptor plugin is missing even if its installed... do any1 else have this problem or how to fix it?
nvm, maybe needed update comfyui... installed as localserver in krita option and moved all needed files to custom installation... now it works
I use Krita for all my professional work. This is great. Thanks for the video. Does anyone know where the control net models go? They work in Comfy, but the plugin says it can't find the controlnet models.
Check the logs - you may have your models with different names
Is there a A1111 version of this please?
Not that I’m aware of as yet, but do let me know if you find one!
Assistant AI Plugin as a mini assistive Drawing for people with Visual impairment, such as I would be extraordinarily benefits, by removing the obstacle-course I had to deal with lapsed Hand to eye coordination, all my life, up till now. I thank you for the tutorial, better get busy creating!
👍👍👍
may i know your computer spec to get this instant reponse from the ai ? i did a try with my computer , but the performance is awful....>.
For any AI stuff at home you’re best with a GPU from the last 5 years. Something like a 980ti will work, but could take a couple of seconds
Fantastic integration. Is possible to have character and environment consistency when you move a skelethon node?
Pushing it a bit with sd 😉
Completely lost me at 5:47. That doesn't seem to be part of the software? I already have Comfy UI as part of the plugin install, so I have no idea what that black screen stuff is? Comfy UI manager? What part of Krita is that? You seem to be talking about a separate software,
As described in the video, you have two options. To install ComfyUI as part of the plug-in, or if you already have ComfyUI installed. The separate software in this case is a standard installation of ComfyUI!
@@NerdyRodent So if already installed with the plugin, how to get to all those options an stuff? *Thinks to himself "Why is everything AI such a headache?"
Yup! You can ignore anything to do with the ComfyUI back end when using Krita and having the plug-in do everything for you 😀 Or, if you want to manage things yourself, go with the ComfyUI install first. The choice is yours!
@@NerdyRodent 2 things putting me off now, 1. it seems Krita developers actively hate AI and want nothing to do with the plugin, and 2. the interface is too big for my 43" 4k monitor (I can't reach the generate button half the time. That's just... ew). I'll try again next year? *shrugs
@@bigglyguy8429 ooh. Nice monitor, and yes… time will tell!
I can't get it to run quickly. It seems to use my ram instead of my vram.
It’s likely not using your GPU then for some reason, as a guess. Make sure you’ve got your gpu drivers installed and check the issues to see if anyone else is experiencing the same thing.
@@NerdyRodent I think it's working now. It's been a headache. I think that what was happening is that, for some reason, the ram had to handle a lot of stuff for a few minutes before the GPU started kicking in. I don’t have an insane system, just 16 gigs of ram and a 1660 ti, and so I am getting about one render every 20 seconds in the live mode.
@@xylonpesquera8605 nice!
🦉
i have a question, can you turn the heads around and the whole character around? it would be great for consistent characters for picture stories!
how to make comfy manager show up???????????????
Make sure you’ve installed it first - github.com/ltdrdata/ComfyUI-Manager
Everything is cool and beautiful... it's just a pity that in Linux Krita's plugin doesn't support AMD GPUs. You can get around this by installing your own Comfy UI server... but it's too much of a challenge for me. There is no A to Z guide on how to do all this.
One a to z guide: How to Install ComfyUI in 2023 - Ideal for SDXL! ruclips.net/video/2r3uM_b3zA8/видео.html - just remember to install rocm PyTorch 😉
@@NerdyRodent Unfortunately, I'm not a software engineer. The only thing I was able to do was install python and rocm, and then I think I ran CmfyUI... all by pasting lines of code from the Internet into the console. Installing ComfyUI manager was beyond my capabilities... and without it I won't upload Controlnet and so on. The "instructions" I found on the Internet are completely incomprehensible to me. I'm a graphic designer, not a programmer 😅
@@veritaspk that’s fine, this is just basic commands that anyone can run 😀 if you need even more help than the step-by-step video, you can drop me a DM on patreon!
Why is it so slow when I generate it? It doesn’t update in real time like you demonstrated.
Older cards like the 980ti will indeed take a second, or maybe 2, longer
Is this plug-in open source? It’s completely free and no signature is required.
My suggestion would be to check the license to see if it’s compatible with whatever your requirements are 😃
Please answer: how to *SAVE RESULTS?* I dont see any options for saving. Where is it?
“File - save” is the typical way after you’ve sent your image over
@@NerdyRodent it saves my drawing, not generated image
@@NamikMamedov yes, it will only save as normal so remember to send whatever you want to save over first
@@NerdyRodent I am sorry, could you make it more clear? Where should I click on generated image to save it? I can't find it in output folder either
@@NamikMamedov tricky to say in text, but basically like I show in the video. The icon is quite smol!
Not working - Error: Connection refused [url= ... prompt, code=1]
there's nothing like this but with a1111? im afraid if i install comfyUI i'll end up with the whole SD+controlNet installed on my PC twice for no reason when someone less ignorant might know how to point the two different frontends to the same models
Symlinks / the ComfyUI config file, are your friend!
it seems for me that it takes a really long time. is it normal ?
It’s about 0.5s render time normally, like in the video
@@NerdyRodent i cannot run it on my gpu, it says it cannot launch the server. I tried updating the drivers of my NVIDIA, it didn't work either. Is it because my gpu is too old? Because when i launch it on my cpu, it launches but do not work. Thanks in advance if you know the awnser !
Yup, something like an old 960 will struggle at best. Something from within the last 5 years is best.
@@NerdyRodent no i'm not this far, i have a GeForce on my laptop it should work. But i gave up and deleted the files to have some Space. It's unfortunate but maybe i'll try again one day
W
What's the purpose of pretending to draw an owl when you can just use text to image and do it instantly
TL/DW: There's absolutely no reason to learn how to draw any longer
"Draw the rest of the funny owl"... *draws cat-bird*
YOU GAVE THE WRONG WAY TO UNPACK THE FILES IN THE WRONG LOCATION!!!! Sorry, it's been a very long few days figuring that out
wow! that's tru-elly amazing'ing! an Owel ! lovely! I bet U could use nVidalia with this one!
& U'v even sorted the req!! what a pal!
When I try to connect to my Comfyui installation I get an error about missing Controlnet preprocessors, but I have them. In the client.log file I don't see any useful information.. any idea? :(
I got the same issue, after trying several types of controller(inpaint), it works.