I can confidently say that the creator of Lora is a genius! It is easier to work with such models, and most importantly, they do not take up much space on the SSD.
At this point I really need a broad overview of everything what I need to know about Stable Diffusion. I am really lost with all this different in-detail topics, because I do not even know where to start.
So basically there is no need for large checkpoints since we can just use these Lora files with the base models ? This will save a lot of space and will make file transfer / downloads faster, even if storage is not a big deal, it helps. Did you do some tests, to see if the prompt with the extracted Lora is almost identical with the original, much larger, model ?
to answer my question.. It kind of works but is not the same as using the original model. You have to fiddle with the weight and with the Network Dimension, which is 128 by default now, and can go up to 1024. But like in the video, higher values give artifacts and a larger file size. Also, some Lora models are trained on different models, not on the standard 1.5, so you need that model downloaded, otherwise it won't work.
I posted a style training guide on Civitai questions section and my own model there. hope it helps some folks. Really huge gaping hole among RUclips creators where style training tutorials should be.
Is there any way to rollback Stable Diffusion to version 1.5? They've discontinued using several artists in prompts which is proving to be quite a challenge for my work.
@@Aitrepreneur as we are able to extract the changes including the trigger words. May be there should be a way to unpack a .safetensor to a diffuser or some format, where we can change/customize trigger words. Programmers who know about how data is saved and handled can help.
@@Endangereds That unfortunately isn't how this works. The trigger word isn't "saved". Text gets turned into a bunch of numbers called vectors. Those vectors gets turned into a text embedding. Those numbers then get added as a step in the diffusion process when training in association with the object being described. All-in-all, once you have trained the model to associate the object with that word, you cannot untrain it to associate with that word. Maybe, in textual inversions case you may be able to create a textual inversion to create a text embedding that looks for the embedding you created in dreambooth in latent space. However, i'm not sure how accurate that would be. It's worth a shot though.
@@hajiamano22 I saw a reply which reads "In theory you should be able to. In the vocab.json file where your model is (model/tokenizer/vocab.json, you should be able to change any of the words to your own, and the word will reference the embedding vector. So looking at the 1.5 checkpoint it would be like: Original: "sks": 48136 New: "my_word_here": 48136 Then just merge the model with the LoRA as you normally would."
@@Endangereds Well, if that's the case and I am wrong (which I can be) editing a json file isn't difficult. You can open it in a text editor like notepad++ or visual studio code and edit it yourself. Test it, and see what if it's true.
Excelent video, I am currently facing an error: KeyError: 'time_embed.0.weight' when tying to extract the LoRA from my safetensors file, unluckily I have not found any solution online. Is someone else facing the same problem?
I'm really confused i don't have the show extra purple button? ive updated diffusion multiple times now and still dont have that button any reason why?
Interesting but as you probably know Luna Lovegood doesn't need a Lora to start with. If you just put it in the prompt you'll get very good results with most models...
Now I can train face but I have a new problem. I want to train clothes Or even rings, bracelets, I can't train it. When I Generate Pictures, It show only the shirt/rings but no face. I've been looking for a solution all week but I'm out of options. Can you help me?
Hi, so if you install Kohya SS on you system does it replace Automaic 1111 as the gui? or how do the two interact together? I am using Automatic 1111 with SD atm but i want to try out LoRA also but don't want to change my current setup as i am very new to AI. Thanks!
It's just another extension, similar to any other extension. I'm not sure about your PC, but based on my experience, it will open a new Chrome tab (different from the Chrome tab that I used to interact with SD). Nothing changes. Although it may require some software update or downgrade.
i'm kinda stupid today from lack of sleep, could someone explain like i'm five, what does this do exactly? Does it mean that if we extract a lora from one of these models , we get all the advantages of that custom trained model with less size? are there downsides?
Less size and you can use it on top of multiple models (and in conjunction with other LoRAs). I'm guessing there may be some quality implications of using LoRA vs Dreambooth (not sure the specifics), but at least some of the Dreambooth checkpoints are because that was because LoRA wasn't a thing yet.
But, to be clear, this does NOT produce identical results for identical prompts?! An image created with i.e. Analog Diffusion V1.0 will look different from (identical prompt and settings) SD15 with extracted LORA from said Analog Diffusion V1.0 Checkpoint. I'm just verifying here - so I can stop trying to make them match and thinking I'm doing something wrong when extracting the LoRA Data. Thx.
So basically the extracted LORA-file is the same as using a Textural Inversion embedding file trained on a subject. So you could just do that... Right?!
@@kallamamran yes it is. And there's nothing wrong with that (don't think I didn't see your ironic tone here lol). Stable Diffusion is about sharing everything to make great stuff; we share the tech, the prompts, everything except the money some of us make from it lol
Not sure this is the right place to send "requests" - but - speaking of merging things together.... Have you tried "Multi-subject-render" Extension? Control net lets you more easily have multiple people now... but I'm still having a heck of a time describing each person separately- e.g. 1 man, 1 woman, 1 child, wearing different styles and colors, and are all different ages and ethnicities. SD gives me some scary results (everyone wearing dresses, or everyone's got boobs) - yada yada. Another thing that might be the key to this- but I think the world needs an easy to understand overview from our AI overlord is ComfyUI -- Node Based SD.... Something Invoke is working on, but hasn't released yet.
This is the bootleg version of LoRA. If you’re extracting a LoRA from a checkpoint, make sure it’s truly from a sd 1.5 base other wise you’re wasting space on your pc
There might not be a LoRA available. Dreambooth came earlier, so a lot of trainings were done with that instead (and there probably is still some quality advantages of Dreambooth, though I'm not sure). This also lets you borrow some of the style of another full model/mix without making your own carefully tuned mix.
HELLO HUMANS! Thank you for watching & do NOT forget to LIKE and SUBSCRIBE For More Ai Updates. Thx
I would love to use the kohya_ss ( gui or no gui ) but it just won't work on my machine :/
I can confidently say that the creator of Lora is a genius! It is easier to work with such models, and most importantly, they do not take up much space on the SSD.
You are the first to bring this method to my attention. Ahead of the pack as always. Thumbs up.
Glad to help
Another first class video from the Ai_Overloard. Your content is part of the acceleration we are seeing with Ai development and enhancement.
I would love a quick segment on what the benefit of loss of different models styles, like lora vs checkpoints.
I eagerly anticipate each of your videos, thank you for such invaluable content!
wow! I've been making a ton of 4GB files. You rock! This is perfection.
Liking the new thumbnail!
I mean, cool but tbh the likeness is not there at all.
This is a winner option 😁
hmm everyone talking about this lora time to learn. :)
So Cool!
Massively useful, effective, kudos [GENERATE].
Is there a google colab for this?
Thanks for your teaching, master!!
This is actually amazing. Thanks so much!
The checkpoint merger does appear to be better however, as you said it comes a cost... A new SSD. lol
This is massively incredible!
Hey, do you think you could do a video walking us through the Kohya lora training google colab? Thank you for your always amazing content!
Very useful! Awesome
Thanks you !
nice! thank you!
thanks for the video. in my case, Network & Conv Dimension in LORA Extract are 128 by default, what is the difference between 8 & 128?
At this point I really need a broad overview of everything what I need to know about Stable Diffusion. I am really lost with all this different in-detail topics, because I do not even know where to start.
This is really cool! Thank you! Also, what GPU do you have?
So basically there is no need for large checkpoints since we can just use these Lora files with the base models ? This will save a lot of space and will make file transfer / downloads faster, even if storage is not a big deal, it helps.
Did you do some tests, to see if the prompt with the extracted Lora is almost identical with the original, much larger, model ?
to answer my question.. It kind of works but is not the same as using the original model. You have to fiddle with the weight and with the Network Dimension, which is 128 by default now, and can go up to 1024. But like in the video, higher values give artifacts and a larger file size.
Also, some Lora models are trained on different models, not on the standard 1.5, so you need that model downloaded, otherwise it won't work.
damm tnx man
Is there any google colab notebook to make lora extraction?
Did you find any? Thanks in advance, I want to do this but I use colab for Stable Diffusion.
Hello everything is fine? Great video! what version of stable diffusion do you use for this tutorial? 1.5?
How would you have 2+ characters in a scene using this technique?
You can use the composable lora and latent couple extensions
Would love to learn how to train art styles! 🎉
can you make a video on a list of the best ai tools you know about like chat gpt or other ai's that you know about
i do hope this doesn't need much vram so that i can extract the lora locally and use the extracted loras with google colab
May i ask what is the difference between lora method and previous textual inversion embedding? thanks a lot
Dude, can you do a video training the Trending dinosaurs with professions or jobs? xD hahaha thanks!
I want to know is there any way to incorporate lora into the big model?
For example luna_lora + sd1.5 merge to luna_sd_model
Did u got an answer?
@@stan-kk2sf so it’s possible to merge a train face of ourselves and a style?
I posted a style training guide on Civitai questions section and my own model there. hope it helps some folks. Really huge gaping hole among RUclips creators where style training tutorials should be.
Can this method extract inpaint model to a lora file as well🤔, so that we can use inpaint model in "lora way"?
Is there any way to rollback Stable Diffusion to version 1.5? They've discontinued using several artists in prompts which is proving to be quite a challenge for my work.
trigger words? ugh. The LoonyLL embedding TI needs no trigger...
why fp16? float is not more precise?
can we change/customize Trigger word at this point?
No trigger words are choosen during the training
@@Aitrepreneur as we are able to extract the changes including the trigger words. May be there should be a way to unpack a .safetensor to a diffuser or some format, where we can change/customize trigger words. Programmers who know about how data is saved and handled can help.
@@Endangereds That unfortunately isn't how this works. The trigger word isn't "saved". Text gets turned into a bunch of numbers called vectors. Those vectors gets turned into a text embedding. Those numbers then get added as a step in the diffusion process when training in association with the object being described. All-in-all, once you have trained the model to associate the object with that word, you cannot untrain it to associate with that word. Maybe, in textual inversions case you may be able to create a textual inversion to create a text embedding that looks for the embedding you created in dreambooth in latent space. However, i'm not sure how accurate that would be. It's worth a shot though.
@@hajiamano22 I saw a reply which reads
"In theory you should be able to. In the vocab.json file where your model is (model/tokenizer/vocab.json, you should be able to change any of the words to your own, and the word will reference the embedding vector.
So looking at the 1.5 checkpoint it would be like:
Original: "sks": 48136
New: "my_word_here": 48136
Then just merge the model with the LoRA as you normally would."
@@Endangereds Well, if that's the case and I am wrong (which I can be) editing a json file isn't difficult. You can open it in a text editor like notepad++ or visual studio code and edit it yourself. Test it, and see what if it's true.
Hi, anyone knows how to train lora which support mask?
Excelent video, I am currently facing an error: KeyError: 'time_embed.0.weight' when tying to extract the LoRA from my safetensors file, unluckily I have not found any solution online.
Is someone else facing the same problem?
how to become pro in Stable Diffusion? I see a lot of beautiful anime pictures, but how do they make it?
I'm really confused i don't have the show extra purple button? ive updated diffusion multiple times now and still dont have that button any reason why?
Interesting but as you probably know Luna Lovegood doesn't need a Lora to start with. If you just put it in the prompt you'll get very good results with most models...
Not sure about the DIM 8 extraction... why not 128 or 32?
i dont find the page where you can extract lora, where is "utilities"? thank you
¿can i use my own face to do this ?
Can't get LoRA to train anything as it claims no xformers found while dreambooth works just fine with xformers.
Auto1111 is turning into a dogs breakfast with all its extensions etc. different cuda/ Python versions doesn’t help either.
@@camclare Different Cuda/Pytorch/Python versions combined with random extention needs makes the system rather un-stable.
Hi im getting an error : KeyError: 'time_embed.0.weight'
Do you know how to fix it?
Now I can train face but I have a new problem. I want to train clothes Or even rings, bracelets, I can't train it. When I Generate Pictures, It show only the shirt/rings but no face. I've been looking for a solution all week but I'm out of options. Can you help me?
I use linux how do I start Kohya GUI?
when i do that i get this error: KeyError: 'time_embed.0.weight'? you know why?
Hi, so if you install Kohya SS on you system does it replace Automaic 1111 as the gui? or how do the two interact together? I am using Automatic 1111 with SD atm but i want to try out LoRA also but don't want to change my current setup as i am very new to AI. Thanks!
It's just another extension, similar to any other extension. I'm not sure about your PC, but based on my experience, it will open a new Chrome tab (different from the Chrome tab that I used to interact with SD). Nothing changes. Although it may require some software update or downgrade.
Will all the lawsuits poping up effect stable diffusions development negatively?
colab tut plssss
How to do the reverse? Merge a Lora into a Checkpoint file?
Bro pease help us make a dteambooth that works even with Joe penna the old dreams booth video you made is broken 💀😭😭
i'm kinda stupid today from lack of sleep, could someone explain like i'm five, what does this do exactly? Does it mean that if we extract a lora from one of these models , we get all the advantages of that custom trained model with less size? are there downsides?
Less size and you can use it on top of multiple models (and in conjunction with other LoRAs). I'm guessing there may be some quality implications of using LoRA vs Dreambooth (not sure the specifics), but at least some of the Dreambooth checkpoints are because that was because LoRA wasn't a thing yet.
@@ShawnFumo Yeah i was having a slow brain day lol. thanks for answering.
But, to be clear, this does NOT produce identical results for identical prompts?! An image created with i.e. Analog Diffusion V1.0 will look different from (identical prompt and settings) SD15 with extracted LORA from said Analog Diffusion V1.0 Checkpoint.
I'm just verifying here - so I can stop trying to make them match and thinking I'm doing something wrong when extracting the LoRA Data. Thx.
So basically the extracted LORA-file is the same as using a Textural Inversion embedding file trained on a subject. So you could just do that... Right?!
This is about extracting datas from existing models. Sure you can make embeddings by yourself but that takes hours, it's up to you
@@tetsuooshima832 So basically it's about extracting other peoples training data :P I get it :)
@@kallamamran yes it is. And there's nothing wrong with that (don't think I didn't see your ironic tone here lol).
Stable Diffusion is about sharing everything to make great stuff; we share the tech, the prompts, everything except the money some of us make from it lol
Not sure this is the right place to send "requests" - but - speaking of merging things together.... Have you tried "Multi-subject-render" Extension? Control net lets you more easily have multiple people now... but I'm still having a heck of a time describing each person separately- e.g. 1 man, 1 woman, 1 child, wearing different styles and colors, and are all different ages and ethnicities. SD gives me some scary results (everyone wearing dresses, or everyone's got boobs) - yada yada. Another thing that might be the key to this- but I think the world needs an easy to understand overview from our AI overlord is ComfyUI -- Node Based SD.... Something Invoke is working on, but hasn't released yet.
how to merge 2 lora models
do you have complete tutorial for beginner with no coding background?
I do yes: ruclips.net/video/VXEyhM3Djqg/видео.html
@@hundredfireify I see
@@Aitrepreneur thanks sir
This is the bootleg version of LoRA. If you’re extracting a LoRA from a checkpoint, make sure it’s truly from a sd 1.5 base other wise you’re wasting space on your pc
i dont understand. why not just download a lora?
There might not be a LoRA available. Dreambooth came earlier, so a lot of trainings were done with that instead (and there probably is still some quality advantages of Dreambooth, though I'm not sure). This also lets you borrow some of the style of another full model/mix without making your own carefully tuned mix.
intro hello msk a wododo
Super video... thnx for this... I like you and your videos... your first AI fan @SALLI.E
KeyError: 'time_embed.0.weight' , I get that when I try to extract the lora. Any idea why that is ?