Comic Character Loras For Stable Diffusion
HTML-код
- Опубликовано: 23 окт 2023
- In this epic tutorial, learn how to train your own character into a Stable Diffusion Lora to produce stunning and visually consistent comic book characters. I’ll guide you through the step-by-step process of using Kohya SS whether you have a powerful PC or a potato.
Discover valuable tips, techniques, and tools to elevate your comic book artistry.
Want to advance your ai Animation skills? Checkout my Patreon:
/ sebastiantorresvfx
To get a copy of the Settings file.
www.sebastiantorresvfx.com/dow...
Related videos ▶️
Consistent Comics With Stable Diffusion: • Comic Characters With ...
Install Stable Diffusion: • Stable Diffusion In Mi...
Consistent faces : • Consistent Faces in St...
Links from the Video ###
www.runpod.io/
github.com/bmaltais/kohya_ss
Visual Studio: bit.ly/VisualStudioinstall
Booru Dataset Tag Manager: github.com/starik222/BooruDat...
Model i used: civitai.com/models/137560/rea...
Jupyter Password
Jup1t3R!
Optimisers
scale_parameter=False relative_step=False warmup_init=False
Thanks so much for these videos. You’re doing such fantastic work for people with stories to tell who don’t have the tens of thousands of dollars to pay an illustrator.
This is simply awesome! Thank you.
Glad to hear it! You’re very welcome 😉
Great video. Thanks for all the detailed steps. 💯
Glad it was helpful! 😃
Excellent
Awesome video series!!
Thank you, glad you enjoyed it. I’m not done yet! 😄
Thank you
You espoused a few things that SAI themselves have said are flat out bad, not awful, just not as good. Don't use tagging use natural language captioning as SDXL was train on NLP. I know, I miss my WD1.4 tagging as well, but natural language really is next level. I searched high and low, and I have yet to find a captioner worth spit, so I fell back to Blip (pretty crappy tbh) and have to go in and hand curate each caption. The next thing is, and I tested this a few times as I was stumped as all get out, is what to caption. I train styles which is even harder, but really do caption what you want to remove. Blonde hair captioned means I can easily make the woman have a different colour of hair. Black pants, white shirt, etc... Anything you don't wish to make a static part of your character, or style, tag/caption it. The last thing is not a biggie so not worth bringing up. Loved the vid.
Hey! hope you enjoy the video. Just a heads up, remember to name your lora in the folders tab. Using the name of the celebrity you selected so its easy to remember. Happy training!!
can u please make a video on How to pause, resume training? How to add more clothes, objects in next resume?
e.g. i want to constantly add new shoes or sun glasses every week. How to do that? PLEAS MAKE A VIDEO...
AI Learners hit this comment to make it requestable.
Thank you in advance from all AI Community
Great video. I am curious what hardware you are running? My training is painfully slow using a 4070 TI and AMD Ryzen 9 7950X. I am currently 1.36s/it which should take 3 days. Any settings I should change?
Have about 70 images. Should I generate 700 regularization images? Also, instead of having repeats at 20, can I set repeats to 1 and just increase number of epochs? With 70 images and 20 repeats, the epochs are really long and not saving enough.
Roughly how long is your training taking? Check out my newsletter on my website. You’ll get the reg images I used in this video for woman and for man.
@@sebastiantorresvfx A couple of hours with a batch size of 4 running on an 8GB GPU. Got your file from the newsletter after subscribing. Looks like the repositories for man, woman are not working. I'm using about 160 regularization images I generated with the same checkpoint that I'm using to start LoRA training.
Try downloading the Reg images again, there was a problem with the hugging face link.
Can it be used to make childrens books too , or do you prefer another model for that ?
Yes you can use the process for anything you need a consistent character for. As for which model go through the cover ai website and search for the style you like, or close to.
@sebastiantorresvfx will you make a video on it anytime soon ? or sticking to comics which is amazing by the way.
Also is it easier to do it using reposer or instant lora ?
which model/checkpoint did you use?
Realities edge XL Anime. Check the description i just updated it with the link.
How to run cmd after giving the WD14 prefix... Could you tell me please🙏
I’m not entire sure what you mean by this? So you’ve captioned all your images and you want to know the next step?
@@sebastiantorresvfx Thank you for the reply 🙏. For me after training the wd14 in the cmd it shows as "winerror 3 the system cannot find the specified path. What can i do☹️
Is there an easy way to get a lot of Regularisation images without having to take them one by one off the internet? how did you get so many?
I generated them all using the same SDXL model I planned to use.
@@sebastiantorresvfx ty
I very much doubt "shuffle caption" helps in any way. When one spent an afternoon preparing super clean captions, one doesn't want that to be shuffled, especially if prefix (trigger) and postfix (realistic, etc) are already in the .txt
Shuffle caption is not an option to use when the dataset is well made.
Try with and without that option, what ever works for you is fine. Also it doesn’t shuffle the prefix and postfix tokens.
Why do we need to pick a celebrity name and not an arbitrary one?
It’s a physical base to start from. Imagine trying to sculpt a face from a block of clay. Takes you a while to get a specific look you’re after. Now imagine you have a classy face pre sculpted and you just have to tweak it to look like the specific face you’re after.
This is purely so stable diffusion knows where to start from when creating your character and isn’t going into the process blind.
@@sebastiantorresvfx Thanks! It makes sense.
Will this deform the celebrity image though, let's say you want to generate an image of that celebrity later, but you have trained the model away from it. Do you just switch model then, or what?
Also using a base celebrity is recommended only for stylized characters or for training realistic faces also?
No problem at all.
So if you want to generate that celeb later on just erase the Lora from the positive prompt and it’ll revert back to the original look of the celebrity.
This method works for both realistic and stylised. Have fun 😃
@@sebastiantorresvfx 😎👍
Thank you, can you consitent lora for sdxl real people?
Thank you in advance
How to pause, resume training? How to add more clothes, objects in next resume?
e.g. i want to constantly add new shoes or sun glasses every week. How to do that? PLEAS MAKE A VIDEO...
AI Learners hit this comment to make it requestable.