7GB RAM Dreambooth with LoRA + Automatic1111
HTML-код
- Опубликовано: 3 июн 2024
- The day has finally arrived: we can now do local stable diffusion dreambooth training with the automatic1111 webui using a new teqhnique called LoRA (Low-rank Adaptation for Fast Text-to-Image Diffusion Fine-tuning). We walk through how to use this new method to create some improved quality images.
Discord: / discord
======= Parameters =======
training steps per img: 150
batch size: 1
lora unet learning rate: 0.0008
lora text encoder leraning rate: 0.00006
lr scheduler: constant
horizontal flip: on
resolution: 768
Use LORA: on
Use 8bit Adam: on
mixed precision: fp16
Memory Attention: xformers
don't cache latents: off
Train Text Encoder: off
Pad Tokens: off
NO REGULARIZATION IMAGES
======= Links =======
Image Cropping Tool: www.birme.net/?target_width=7...
LoRA Reddit Post: / p_using_lora_to_effici...
LoRA Github: github.com/cloneofsimo/lora
======= Music =======
Music from freetousemusic.com
‘Animal Friends’ by LuKremBo: • lukrembo - animal frie...
‘Onion’ by LuKremBo: • (no copyright music) l...
‘Rose’ by ‘LuKremBo’: • lukrembo - rose (royal...
‘Snow’ by LuKremBo: • lukrembo - snow (royal...
‘Sunset’ by ‘LuKremBo’: • (no copyright music) j...
#LoRA #stablediffusion #aiart #art #ai #techtutorials #tutorial #dreambooth 
Наука
![[Tutorial] Create Your Own LoRA | Easy and Free! - Google Colab](http://i.ytimg.com/vi/Un9SHPVAAbE/mqdefault.jpg)
![Kehlani - Next 2 U [Official Music Video]](http://i.ytimg.com/vi/MpI7ekFG8A0/mqdefault.jpg)
There's another really good video on this topic btw: ruclips.net/video/gw2XQ8HKTAI/видео.html&ab_channel=NerdyRodent he does some really good ablation on different params, hard recommend
Is there any cpkt files that have built in dreambooth? Where can I find the ones that have dreambooth as part of it
Finetuning with SD can be quite overwhelming. Thank you for your tutorial about LORA :)
Thanks, I'll try it tomorrow. Everyone have a beautiful Christmas!
Thanks for the rundown!
Your explanation was the easiest to follow and resulted in immediate success. Thank you I have subbed :-)
ChatGPT PROMPT: someone just made a nice comment on my youtube video """
Your explanation was the easiest to follow and resulted in immediate success. Thank you I have subbed :-)"""
write a nice reply thanking them
REPLY: Thank you so much for the kind words! I'm glad that my explanation was able to help you and that you were able to achieve immediate success. It's always a pleasure to know that my content is able to make a positive impact on others. I really appreciate your support and subscription, and I look forward to continuing to share helpful content with you in the future. Thank you again for your kind words and for taking the time to leave a comment.
I trained a model for the first time using this vid. Thanks!
amazing, well done mate!
Was waiting for this one.
Thanks for making these. Also, I think tintin is tired of playing around, we need to be alert.
watch out Rastapopoulos
@@lewingtonn ha
can't get this to work i have rtx 3070 8gb but i keep getting "Exception training model: No executable batch size found, reached zero."
Thats awesome. I have a few rendered out 3D models, that I no longer have the software or time to use them properly. I am thinking about exporting them all out as .pngs, then the character will shown at every angle, but only one pose. Do you think that would make a good accurate model to re-vamp the character?
THANK YOU! I finally got it!
There is a 2.1 512 version model too, so you can Train with that one instead of the 768.
My training keeps coming out wrong. I'm trying to do a person but when I test the model it gives me a baby, bird or mattress. Any suggestions on what I need to do?
ur videos are so good
what software do you use to record your videos? i like the rounded corner and layout. is it OBS?
Would adding captions in text files corresponding to each training image make the model even better? e.g. ''tintin/a man/custom identifier walking outside in a blue jacket''
thank you g
working good
on training, I got this error: Exception training model: 'type object 'LoraLoaderMixin' has no attribute '_modify_text_encoder''.
any thoughts?
Bravo, great video!!! What kind of graphic card do you use ? With my rtx3080 it doesn't work above 256X256 resolution.
I am so lost as a lot of new features now, while a lot of this is now obsolete (removed) so this helpful video is now obsolete.
Is dreambooth broken or something ? I'm able to do training but my model makes no difference at all for my prompt. And the preview are nothing like the images I gave it.
Bro please help, as soon as I start train, it says import of xformers halted; None in sys.modules
Fuck yess. Finally a tutorial that worked for me
Ey! I made your tutorial and I had the same problem. It only trains 500 step. Did you know what was the problem? Thanks!
Awesome work!!
eeeey bass! merry Christmas ya drongo!
@@lewingtonn LOL mate a big Merry Christmas to you to bro! Hoping you have a good one with the fam :) as always love your work!
@@lewingtonn Oh actually a question, I am on a Mac not sure if you can run the xformers on it however If I am trying to train a STYLE what would the filewords and prompts actually be?
@@TheCopernicus1 If you're training a style you'd still use normal filewords (e.g. the style is monet, you'd still put "an impressionistic image of a bridge" or something like that)... and you don't NEED xformers (except for some 2.0 models), it just might take more mem
@@lewingtonn Thanks my friend!
Awesome😁
Thanks!
Thank you for this tutorial, but I can't make my own model :\ I get a long complex error that ends with tuple index out of range when I try to create a new model :(
my version of Automatic 1111 says "settings" instead of parameters and seems to be missing some options you have. Also when I installed dream booth it had an error saying i did not have permissions to install lora but I can see lora after rebooting Automatic 1111. I matched the settings as best I could and got Returning result: Exception training model: 'No executable batch size found, reached zero.'. and I am finding nothing online that an help me. If anyone has any Ideas please let me know.
I'm getting the same error, if you got it running please let me know
Man, I don't know, how to make this Xformers thing installed on my PC (windows 10), my automatic 1111 when boots, whos that it has no xformers, I have only either Default or Flash_attention, I tried to install it with Anaconda, I get error in the end. I tried CUDA 11.7 1.8 12, I'm on python 3.10 (do I need to get 3.9?) with all damn visual studios and stuff, still no use. Why it's not working at all? should I maye get another automatic 1111?
Today I used kohya-ss and could train a lora model of 512x512 (and 32 dim, 32 alpha) under a 6GB vram card.
But I could not manage to do so in auto1111, so I think the later is more bloated and that makes a difference when every vram MB matter!
I would love to see a video that explains how to combine a style checkpoint with a character checkpoint so that one can generate an image of a person/character in the style. I have had success creating checkpoints for styles and characters separately, but the checkpoint merger has never worked for me to create a model. Ditto with Loras. I am hoping there is an alternate solution.
"Clone a model for training" ... I was looking for "how to train a model from scratch" basically. But I guess that's impossible right ? I have to start with SD1.5 model or any model downloaded around, in order to train my own images. Right ?
yeah lol, you'd need like $50,000 to train one from scratch... come back in 12 months tho...
Hard, not impossible. Keep in mind that Ai been around for some years and most of that time have been spent downloading images from sites and then train the Ai on them, and that been done by universities and companies, to do the same alone would take very long time if you want same result. So it best to just train on things you are interested in to use in art, may start with their own face and then goe over to what they like or want, like art and cartoon styles, a location and so on.
But do not give up, the technologo rush forward and in the future there may be tools to help single users to build more complex models, Ai creation need folk that are villing to put in time to create models.
when lora got released i was amazed at how much people can optimize these things. Training seems to be the next big thing compared to prompt engineering.
hopefully i can run it sooner or later when it becomes available using a 2060 (6gb vram)
you might be able to run it @ 512x512!
@@lewingtonn hmmm wouldn't that cause some distortion? or is there a 2.X model thats designed for 512? Either way, I can always do it on runpod as I've done on my other models :)
LORA for Stable Diffusion Video on youtube by Nerdy Rodent says it works on 6gb cards, Does show him uncheck the fp16 and use none (he mentions thats sometimes needed for older cards)
@@oldman5564 My 1060 does fp16 and many have been using it with it checked at 768x768 with much better results than him. Now pre 10x0 cards you don't have a choice and must pick none as I do believe the cards prior to Pascal couldn't do any floating point (I can even do fp32 with mine but omg is it ever slow and the vram issues).
@@RexelBartolome yes there is a 2.0 512 model.
finally cracked it... for best settings to insert yourself via LORA dreambooth, use 10x the number of pics of you youre feeding it as class images, LORA learning rate 0.0009, LORA text encoder learning rate 0.0002, LORA ticked, 8bit adam ticked, fp16, xformers, Dont cache latents or pad tokens. Every guide i've read or seen so far has the text encoder learning rate set way too low at like 0.000005 and it does nothing but waste vram that low. I cranked it up and it fixed my problem.
512x512 max for 5-600 steps per image. I've tried higher resolutions but unless you have more than 8gb, the highest that will train text encoded is 704x704 but it doesn't work. Like it says it trains but it doesn't really. 768x 768 with no text encoder or 512x512 with it but for a person that doesn't already exist in the model, it's important to include it. Gonna try 640x640 as well but I don't hold high hopes for it after the 704x704 results. 512x512 works pretty well with 1.5 versions and it worked great with the 512 base version of 2.1 when I tried it so who knows 🤷🏻♂️
When I click on Training model/person, first it gives me a warning and then raises my steps to 150, if i click it again then it changes Image generation > Class Images per instance image to like 16. Every video i watch this is in the 100's, but if i set it to like 100, then it has to go through over 2000 images. Also my models are coming out not looking like me
Trying this at the moment, with an 8GB 2080, and the newer version of Dreambooth - does not work.
Help. I get this message:
Returning result: Exception training model: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 8.00 GiB total capacity; 7.26 GiB already allocated; 0 bytes free; 7.30 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
yeah i get the same cuda out of memory error and also have 8gb vram, very confusing
@@JohnEliot1978 10GB and getting the same error. :(
did you fix the problem ? I've 12 gb and get the same
I get "join() argument must be str, bytes, or os.PathLike object, not 'list'" error after webui finished generating class images.
using 3060Ti with xformers
im getting the same error with the same GPU
any chance you could update your guide now that it doesn't work with the new version using 8gb vram? pretty please :)
Isn’t it so annoying! I just tried to run it and only just found out that it doesn’t even work for 8gb because if the update 🙉
sorry, so source of checkpoint would be what?
Still not able to run dreambooth + LORA in my rtx 3080 without getting OOM exceptions :( No clue what is wrong. Even turning eveything off and just keeping 1 monitor
that's super strange, do you know how much memory IS being used (e.g. performance tab for GPU on task manger)
@@lewingtonn Using nvitop to get the memory usage before launching SD i manage to get to about 300mb VRAM, killing everything and unplugging one of the monitors.
After starting to train, it skyrockets, and the exception says it had reserved about 9.2 and had 0 free.
@@RemusRichard yeah, ok then it's a very straightforward too-much-ram issue. For me it only took 7 GB so there is some kind of difference in our config. I know that the mixed precision setting is very important so is use 8bit adam... possibly you might need to install xformers too (ruclips.net/video/ZVqalCax6MA/видео.html&ab_channel=koiboi)
@@lewingtonn installing xformers didnt solve the oom errors for me either :( also have 8gb vram
i'd also like to add that if youre using this method to model a particular person, i'm making one of myself right now for example, you need to generate class images but the wizard options under the LORA box make this easy by filling in an appropriate amount of classifiers for the task by clicking one. Be it person or object. without generating them, it wasn't really able to add my face to other characters like a Colab generated Dreambooth otherwise would but I have a feeling this is what I was missing from before... ill update in the comments if my theory holds
I was wrong about this. It's the training of the text encoder that allows for better prompting but at the cost of more vram overhead. My RTX 3060 tie fighter oomed on a 768x768 set but it's chugging thru a 512x512 with text encoding enabled
A ghetto workaround for this would be to train a 512x512 model with the text encoder ticked on then training the same model again for a duplicate number of steps with 768x768 and no encoder 🤷🏻♂️
After some tinkering, if you're training a person, use 10x concept images as the number of source images, 300 steps per picture, lora learning 0.0009, lora text encoder learning 0.00005, polynomial or cosine learning rate scheduler instead of constant(box still unticked), text encoder checked, 704 resolution, no other programs eating vram, and she'll go...
Nice one...still not sure about the whole mess of merging the LORA back into the checkpoint, it would be awesome if this would work kinda the same as embeddings work!
And Merry Christmas to you toooooo!!!!!! 🎄✨🎁
haha thanks, really though, it's very easy to merge back, just a click and then like 20 seconds of processing
@@lewingtonn true, but the thing I really like about embeddings is that you can change them on the fly...without having to reload the checkpoint.
But of course they are also less powerful than LORA or Dreambooth...
@@swannschilling474 huh, I wasn't aware that frequent swapping was a part of anyone's workflow, interesting!
@@lewingtonn there are a lot of new embeddings popping up right now and you can also use multiple embeddings or merge multiple embeddigs into one...so when prompting it is very easy to change an embedding vs changing the checkpoint!
That is how it was supposed to work but Automatic never did it.
can someone please help me. i try to run the dreambooth extension locally on a rtx 3070 laptop 8gb and i always get the same error. it says cuda out of memory, but from what I've gathered it allocates 0,0GB although ther should be enough to allocate. I'm not a programmer or understand anything I'm doing but a musician trying to use SD for music videos. So please talk to me like a dummy and specify exactly what to change in what file or setting.
thanks a lot in advance ❤️
what are you doing exactly? just normal dreambooth training? I would try checking (1) are your images too large and (2) what does the task manager say about your GPU RAM usage while training vs at rest?
@@lewingtonn I made a folder with sample images for the instance that are 512x512 and try to create a ckpt to make stable diffusion able to draw members of my band. I also tried both to let dreambooth automatically make the sample images to compare to and also to do it beforehand and designate a folder for dreamboth to load from. i pretty much tried every combination of source checkpoint, lora no lora, adam, xformers etc. the problem starts at the very beginning because it always allocates 0.0 GB of Vram and and finishes training without doing a single training step. Also I'm on my third clean install of both SD and the dreambooth extension.
@@lewingtonn i will check my taskmanager but from what I've understood from the Errors I never even got to training.
@@niknitro8751 hmmmmmm, perhaps you have too many images (it might be loading them into memory all at once, but that's doubtful). One good debug is to make sure that you can do normal image generation at least, and see how much RAM that takes up
@@lewingtonn Normal generation works fine. also hypernetwork works. how many images do you recommend? should I just try with one single Image for testing it out?
Error:
Exception training model: 'No executable batch size found, reached zero.'.
Hey does anybody know why i cant create a Lora that's worth like a few 100MB's at most? Whenever i create a lora from a base model my output results in a multi GB filesize. Please help!
Still having an OOM when saving.. I checked my files and I have the two fixes that have been merged.. 🤔
Tried to allocate 16.00 MiB 🥲(GPU 0; 6.00 GiB total capacity; 5.19 GiB already allocated; 0 bytes free; 5.26 GiB reserved in total by PyTorch)
Any clue ? What parameters to reduce, to reduce the Vram usage when saving? It was 20 MiB, I checked the "don't cache latents" and it went to 16 .. almost there
hello friend, i have updated the dreambooth version and the interface has changed, now i can't train any more, i have a gpu quadro rtx 4000 8gb vram and before the update yes i could train,
I have the same question. I hope someone can upload the old version.
Or someone can tell me how to set up the new version to train normally with 8G VRAM.
you got the error out at 500 steps bc for whatever reason, even if you leave it unchecked, it attempts to save a checkpoint and generate an image every 500 steps unless you set the Save Checkpoint and Save Preview frequencies to 0. Hope this helps with future trainings!
thaaaaaaaank you!
I remembered this step from the dreambooth training video by Olivio Sarikas and it fixes the problem on LORA as well. Up to 1050ish so far on my first attempt after remembering that when mine oomed and 500
Also checkout the Dreamlike Diffusion model sometime if you haven't already. I'm really loving it so far and produces images quite similarly to midjourney. It's a 1.5 base but I honestly haven't been all that impressed yet by 2/2.1 bc of the added requirement to run at full precision if you want it to work properly which doesn't go over so well on 8 gig cards as I've found out. Either way... Dreamlike diffusion 👌
Merry Christmas from my side of the world to yours!
@@RanDappProductions sounds mint, I'll check it out
What is the number of class image per intance for a man
Australian accent, are you wearing that hat in the middle of summer 🤨😄
Koiboi, waiting for Krita extension v2.0 with impatience.
There are so many good webui's these days that I don't even know if krita is worth it tbh :(
@@lewingtonn But with Krita, one can actually draw. Mixing traditional art and AI generated images may makes the life of artist easy. I think it's a workflow that can speed up results exponentially. For instance, I can draw the characters and then generates better coloring and backgrounds. I don't think outpainting webui's can do that.
Still the only dud that makes stuff work . But he only gets 4000 views.
quality audience > quantity audience mate
ya i just get > Exception training model: No executable batch size found, reached zero.
I am doing this but it says its doing 56700 steps. It will be a while it seems. what did I ddo wrong I don't know because I did exactly as you did.
you must have many more images than I did, I only had 12
@@lewingtonn i have 18 images only. But I used regularization images and it was multiplying that. I dont know why though.
@@AgustinCaniglia1992 yeah, short of reading the code, or looking at the automatic1111 github issues page, you might just have to wait for an update or manually stop it
@@lewingtonn I solved it now. I did a working model although the resemblance isn't good. Trying again with new settings.
@@AgustinCaniglia1992 what was the issue in the end?
Finished and it never saved the lora stuff. edit: It didn't because no matter what I do it wants 20-30mb more before it saves. It does train it though.
Hello, soviet comrad! Where is your balalaika?
I have a bouzuki, is that close enough?
12 images x 150 steps = 1800 steps, if you trained 501 steps that means LORA stopped working after 3.3 images
How many steps per image?
Problem is you forgot negative prompt
yeah, I'm still working out how to use those properly... Is there a good guide somewhere?
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
pip install -U xformers
this is what its saying me to do
Dreambooth extension has updated, so...... :P
can u just use the pt file?
no sir, you cannot, but the conversion is very easy
can you make lora in google colab?
As expected: "Exception training model: module 'tensorflow' has no attribute 'io'"
has anyone else been unable to get anything but green and black results out of 2.1 on a 6gb card?
Think the black screen is from when it runs into a nsfw not sure what green screen is from.
wrong paramotor or bad ckpt might be
@@aDAMoscar just using the standard 2.1 512 ckpts
@@oldman5564 don't have any nsfw filters, running locally
@@springheeledjackofthegurdi2117 im using the 768 had alot of black screens ususlly restarting the whole thing and currently using arguments --no-half and --xformers and its working after the last git pull. but i never know when it might go black again
I'm an artist and I have several thousand images. They're finished, but I also have thousands more sketches that need to be finished. I need to train an AI to finish my sketches into completed pictures
Another easy-to-follow tutorial, but could you include your computer specs please? Your 'small potato' may still outweigh my serving of French fries.
It's depressing how quickly graphics cards with 8GB VRAM are becoming obsolete. It's not just in the AI space, either. 8GB of VRAM is also restrictive when working with 3D programs like Daz Studio.
10GB RTX 3080 were such a rip off. It's so low for a "4K" graphics card. I barely manage to make stable diffusion work with that.
It is annoying that all these training methods (TI, DB, hypernetworks) have a ton of these magic numbers and tokens and whatnot, it would be nice if there were some at least some tooltips and better yet, docs for these, now it seems like all the articles and videos have their own idea what to put in these fields...
I usually find that the automatic1111 wiki is a pretty darn good place to start but I know what you mean, sometimes you have to trawl reddit for some good hyperparameters
2:48
I must say, I am very disappointed by LORA fine-tuning. I was waiting for its implementation for weeks and the results do not live up to the hype. The original Dreambooth is perfect in my opinion, and I just wish it could work on my RTX3070ti with it's 8GB of Vram.
You can just alter the settings seems to run out of memory around 3000 steps with unaltered settings on 512/512 (3070nonti) 2000 steps ran to completion after a few settings changes.
LORA for Stable Diffusion Video on youtube by Nerdy Rodent says it works on 6gb cards so your 70ti should run just fine. His vid goes into more detail. Might help ya set it up
@@oldman5564 I watched it first and tried, and I had no issues with memory. It's just the results are... Meh! I'll wait for an even faster version of dreambooth.
You can already use Dreambooth with no GPU at all
@@mikealbert728 Have you tried it? It takes about 19 hours on my PC. The problem with CPU fine turning with DB is it's it runs on a single core! It takes forever.
Activate Windows please.
lora is pure sheet, my trained person models don't have anything in common
Please activate your windows😆🤣
hmm... maybe I'll start a GoFundMe...
@@lewingtonn I think you should. Keep up the good work and I am sure you will be rewarded!
@@lewingtonn lol you get win keys for as little as a few pints these days on ebay! ;)
@ramanauskiene edita hehe if only it was easy as that lol
trying to understand why my trained lora shows up with a lora.safetensors file extension.
На нем что ушанка? Хоть не руzzкая? Слава Украине русаки