How To Do Stable Diffusion Lora Training In ComfyUI (Tutorial Guide)
HTML-код
- Опубликовано: 23 сен 2024
- How to use the new Comfy UI custom nodes for effortless Lora training. In this video, we walk you through the installation process and demonstrate how to train Lora easily using the provided custom nodes. With Comfy UI, you can simplify the process and save valuable time.
Resources For ComfyUI Lora Training : / 100723165
Lora Training Using Kohya-ss and Colab By Dave : • Stable Diffusion Train...
The tutorial covers the essential steps, including installing the Lora training Comfy UI custom nodes from their GitHub page. We explore both the simple and advanced modes, showcasing the range of options available for Lora training. Additionally, we highlight the image captioning custom nodes and guide you through the installation process for generating image-to-text prompts.
Throughout the tutorial, we provide a step-by-step demonstration of the workflow, including loading captions, showing text, and saving text strings in the desired destination folder. We also discuss the WD 1.4 custom nodes, which enable image processing as text prompts, and explain how to connect them in the workflow.
By the end of this tutorial, you'll have a clear understanding of how to use Comfy UI custom nodes for Lora training and image captioning. This time-saving method will revolutionize your Lora training process, making it more efficient and user-friendly.
Join us as we explore the power of Comfy UI and unlock the potential of Lora training. Don't miss out on this easy-to-follow tutorial that will enhance your workflow and boost your productivity.
If You Like tutorial like this, You Can Support Our Work In Patreon:
/ aifuturetech
Discord : / discord
Good stuff! I watched at least 20 vids yesterday, and none were clear and thorough. You, sir, deserve more subscribers
Appreciate for your support , hope this video help you guys. :)
@@TheFutureThinkercrap, never mind. The new images were jpg and not png.
I created my lora training with Lora training in ComfyUI I cannot see my lora in models/loras
This node does not work. I can not get it past the first step then it stops and will not work again. Any reason for this? is there a specific model you have to train in? doesent seem very useful at all if it only works with one model or not at all which seems to be the case for me. I have watched many tutorials on this and none address this issue
Same issue, Video dose not cover this common error, i guess i keep looking
after trainnig i dont see a my new lora in the lora's folder, that's strange
this node broke my comfyui unfortunately. i tried to repair but math nodes ware irreversibly busted. ( make a clone of your setup - if you want to experiment ) just saying.
Did mine too. I recommend you pip install packages from scratch and updating your python to the latest version. It solved my issue.
Thank you for tutorial, this is more convenience to train lora.
Yes it is, just simplify the process of training lora model. In theory, it can also do it for CP model, i believe.
I got error 126 "fbgemm.dll Error loading" after installing that node
Thanks for always giving us the latest methods, i'm gonna try and see if i can improve my LoRa's with this.
Good luck! And have fun with Lora :)
Hey thank you! Im able to do evrything but after training the model I cannot find it anywere... it seems its not doing the tasks also it takes its time and doesnt come up with any errors..
keeps uninstalling torch and reinstalling one without cuda then complaining it wasnt compiled with cuda, gave up
tried installing your nodes and my comfyUI crashed and don't open anymore, will have to reainstall everything...
1- That is not my node, please clarify.
2- if you follow the installation, it wouldn't be a problem.
3- Crash Custom Node can be deleted in folder and ComfyUi reopen again. It can run normal. Shame so called AI on your name, and don't know how to do troubleshooting.
@@TheFutureThinker I downloaded the workflow, installed the missing nodes in the manager, clicked om restart and it just got stuck, I closed and open again, and it stops working, comfy dont start anymore, I updated everything but nothjng. Ill just reinstall again from zero.
@@tonon_AIit happens sometimes. It's weird like that. So best look at issues page before installing new nodes to see what problems others have
"Found a solution that worked for me and other people as well." you might try.
Go in to your ComfyUI_windows_portable then Update and use the Update comfy and python.
Also took away this Node for the Training of Lora.
Yup the installation of the first 'Lora-Training-in-Comfy' and restart after also completely wrecked my Comfyui. I was unable to start it. Gives the following message in the terminal: "AssertionError: Torch not compiled with CUDA enabled." Deleting the crash custom node folder doesn't fix the problem.
EDIT: Updating with 'update_comfyui_and_python_dependencies' in the Comfyui>update folder fixed it again.
Hey, thanks for your explanation. But I struggle on something when I try on my end. I've created a clean Comfy UI install, just installed "Lora-Training-in-Comfy" but at the restart moment, I get the error "Torch not compiled with CUDA enabled". I don't figure how to get rid of it with the info I found on the internet. Do you have an idea of what I should do ? Have a good day
I had the same issue, saw a lot about ensuring you have pytorch and cuda 12.1 which is probably correct however I couldn't get it to work until I did a fresh comfyui install and moved the comfyui portable folder. It has to be the first folder in the path like this E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy
I would try this just to make sure, the issue is with a folder in the huggingface folder with a very long name during install
@@vibeactvst8686 Hey man, i've tried some random things based on your comment and thanksfully it worked after some attempt. Thank you !
@@alexandretamisier8150 Happy it helped somehow. Now I am having issues with rust compiler/cliptextmodel/transformers
One of those isn't letting me training the lora 🤦🏿♂️
@@vibeactvst8686 Same thing, i don't know the reason but the lora training stop instantly when i want to run it :( i've tried to install Kohya too and get errors, i think i should learn python before trying this haha
Thanks for your tutorial. but I've got an error. 'C:\StabilityMatrix-win-x64\Data\Assets\Python310\python.exe: Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')' , could you let me know how to fix it?
Have you recovered yet?
Hello, same for me! Not sure what my options are?
Why is your Python running in Stability Martix? Seems like it doesn't work with Martix installation
Is there any solution?
hi, same for me, and i watch many time from video of owner (Larry Jane). but It still not work, if anyone have solution, i will I would be very grateful.
Thank you very much for the tutorial, but this does not work for me. :( I can make the text files for the images, but "training" takes 17 seconds, and no file is produced in my loras folder. I get no error, the terminal just says it was completed, but there's no file there. Can you tell me what I'm doing wrong? This should be simple, but it's NOT. :(
This node has never worked for me. every tutorial seems to make it look easy but there is not really detail ever shown of the actual process. I see no usable models ever demonstrated after creation by any of the posters of these videos. maybe someone will actually explain why it fails for so many people.
@@Halfgawd_Halfdevil I finally got it to work by making sure that the folder that held my database folder was at the root of its directory ...IE, "D:folder/database folder/". If you try to train the files to anywhere else, it won't dork ...or I guess I should say it didn't work for me. Also make sure that you name the main folder "database", and name the folder with your actual database in it "5_s" and then the name of your LoRA, with no space in between. So, my folder was called "5_sNPC", because the name of my LoRA was "NPC".
By the end it was set up like this: "D:database/5_sNPC/".
Hope this helps!
@@RuinDwellerGlad you got it to work! I have all the database folders etc. but I was using "5_" without a lowercase s. I will give that a try. I have no real issues with the tagger saving to the database folder with only the 5_ before the model's name so not sure why one would work and not the other, but I will give it another try. Thanks for the info. I appreciate it. BTW how are the Lora's working? Do they take long to train...so I have a reference for time I should be waiting? Is It still sd1.5 only? just curious as I use almost exclusively SDXL so If it isn't going to train in SDXL it's really pointless for me anyway.
@@Halfgawd_Halfdevil I didn't try with XL simply because it has a higher VRAM requirement, it's even slower than 1.5, it already took me 20+ hours to train 75 epochs, and it scraped the ceiling of my VRAM the whole time. ...But it's possible to do! You just have to have a much better system than I currently have.
Did you get this to work?
I did get mine working, but even after all that training, they still aren't really "crisp" or photographic, but I probably need to turn the training up. That's another thing that's missing from all of these tutorial videos. The settings they show before training SIMPLY CANNOT be the ones they actually used for it, there's no way.
Introducing an Advanced node and Access Tensorboard node! Access Tensorboard is a very simple node that launches a URL to see data about the logs created during training. Click the node once (after, during, or even before training!), then copy-paste the URL that it wrote in the command prompt. The logs folder related to Tensorboard is in the same folder as your launcher file. (I don't understand this part and where to copy the URL, can you help me with this part?)
I have it working at 5 epochs and 10 batch but I keep geting out of memory errors when I go to a batch count of 100 or more
not sure why I have an RTX 3090 24GB ov vram
also need result demo
Did anyone get it to eork with SDXL models?
Gonna give it a try this week.
Training is not working, can you please update
Hi bro, i love you work,have a great day, i have one question ,This method work on amd gpu?
It depends you Comfyui setup. I haven't play with AMD GPU yet. So I can't answer you accurately
what about the files preparations? file sizes, nomenclature, proportion etc anything goes? I miss some of that infortmation. thx.
it's called Dataset. SD 1.5, and SDXL image requirement are difference, it's 512X512 and 1024X1024 px.
I don't explain everything about Lora, this video are only focus on this custom node. If I need to explain everything about Lora, it will need a hour long video, and I don't think you have the patient to watch those. ;) cheers.
@@TheFutureThinker Well then this video is not about "How To Do Stable Diffusion Lora Training In ComfyUI" but instead it's how to understand these custom nodes for people who already have some baggage in lora training.
I keep getting an error that the node in costom-nodes/huggingface/__init__.py doesn't exist, I searched everywhere but I cant find if there's a thing as this custom node. I've also installed the requirements but it doesn't install this things. Heeeeelppppp
Okee, I fixed that one by putting an empty __init__.py in there - Up to the next error lol
Um.. in my setup, I haven't use Huggingface node.
@@TheFutureThinker I've tried about everything cant get it to work. Installed torch more times then I can count, tried in vr anaconda. Checked all the cuda stuff with different version... Alwayst get error related to some dll or some ini file. I give up lol, wasted way too much time
Yes, if this don't work , just use Colab Lora training or Kohyass. Some open source project is like that, they created it but not compatible for every system.
@@TheFutureThinker Yeah thanks for the video though, Im using Kohya now, seems to work on my system
OK, you sold me on comfy. I've been training with kohya dreambooth and onetrainer and love trying new training tools. I can't get sdxl to train in onetrainer at all but I'm only on a 3080 so maybe the vram is lessened here.
I used Kohya dreambooth to lora lora and checkpoint before. And when i saw this custom node, i feel it is very convenience
@@TheFutureThinker you said after training in the end how do we train? I do not know what to click to train 😅
@@galdren_cyphr click Queue Prompt in Comfy as usual to execute workflow
Can you share workflow
Its in the folder.
I get this when i try i use the trainer Error while finding module specification for 'accelerate.commands.launch' (ModuleNotFoundError: No module named 'accelerate')
it killed my comfyui, i wont start anymore. i only tried the first node with the manager. RIP.
same with me
Hi, Ive been having a lot of issues with this and I've gotten so close to fixing it all. When I start the training everything works well until I am met with this eror:
raise RuntimeError('Error(s) in loading state_dict for {}:
\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIPTextModel:
Unexpected key(s) in state_dict: "text_model.embeddings.position_ids".
I dont know how to fix this and I cant find anything online that helps.. Hope you can help me out with this! thanks! (:
anybody els get stuck with the error:
'pip' is not recognized as an internal or external command, operable program or batch file.
when doing the cmd: ComfyUI\custom_nodes\ComfyUI-WD14-Tagger> pip install -r requirements.txt ?
try to type all: ../../../python_embeded/python.exe -s -m pip install -r requirements.txt
it works for me