The BEST, Local Text-to-Speech Generator - AI Voice Cloning (Tortoise TTS)
HTML-код
- Опубликовано: 23 авг 2024
- If you wanna take a look: git.ecker.tech...
Running the Voices - 3:00
TTS for LLMs Talking - 7:15
Audio Book Reader - 8:15
Come join The Learning Journey!
Discord - / discord
Github - github.com/Jar...
TikTok - / jarodsjourney
If you found anything helpful, please consider supporting me and the content I am trying to produce!
www.buymeacoff... |
Hardware for my PC:
Graphics Card - amzn.to/3pcREux
CPU - amzn.to/43O66Ir
Cooler - amzn.to/3p98TwX
RAM - amzn.to/3NBAsIq
SSD Storage - amzn.to/42NgMFR
Power Supply (PSU) - amzn.to/3NBAsIq
PC Case - amzn.to/447499T
Mother Board - amzn.to/3CziMXI
Alternative prebuilds:
Corsair Vengeance i7400 - amzn.to/3p64r22
MSI MPG Velox - amzn.to/42MnJHl
Cheapest and minimum specs recommended:
Cyberpower 3060 - amzn.to/3XjtZoP
Glad this sort of thing is available locally. A lot of the AI stuff is awesome but 11labs for example has pretty steep character limits if you really want to mess around. Messed around with the starter version of 11labs for quite a while and the 40000 characters per month really isn't that much
I think we're not far from being able to run all the crazy AI models out there locally, I mean of course a supercomputer can always work better but at least we're not doomed with api token costs for everything
Totally agree, I love the quality of 11labs, but if you're innovative with local stuff, we can now get very close to 11labs quality.
Wow, so glad you made this! I used Tortoise a lot a while back with the original colab. I LOVE that there is a new interface. This is the find of the day.!
Appreciate it! The GUI fork opens up a lot of knobs and values previously hidden, so it's definitely pretty awesome in my eyes!
@@Jarods_Journey took me a bit to get it going, but for whatever reason, all of these installations take forever on my system, even though I’ve got a fast GPU and lots of RAM. Anyway, I’m awaiting your video on how to train, meanwhile I am using the quick and dirty utilities to create quick voice clones and it’s working great.
@@BobDoyleMedia that's awesome haha! If you're already training voices, I'm not sure I can provide much more additional value other than what's on the GUI. If anything, the author was so gracious enough as to writeup what mostly everything means and it's quite comprehensive. I'd take a look at that id I were you to get a head start!
@@Jarods_Journey Well, I'm not so much training voices as I am dragging and dropping a :60 clip into the Utilities area and creating a quick voice like we did with the old Tortoise. I haven't trained anything with the Train tab.
@@BobDoyleMedia Gotcha gotcha! Well tutorial soon, so we'll see how that goes
Great video man. Inspired me to jump into Tortoise and make a short reviewing it and teaching people how to set it up. I just used demo voices, but your examples are amazing.
Glad I could inspire ya, the short, 1 minute tutorials being nice and concise are perfect xD!
The one used here is a forked version of neonbjb's that incorporated training into it, allowing for the examples you heard.
Awesome video! Can't wait for the guide.
Hey man, can you do more videos on the different/best ways to do Speech-to-Speech voice cloning, and how to train custom models? Some of the current workflows are kind of complicated/inconsistent, and I'm hoping that the tech becomes more streamlined in the near future with better GUI's and workflows.
The workflows are kinda clunky right now and when errors are encountered, it's evenso much harder to debug, but some things are in the pipeline. Currently testing and working with RVC right now so that'll be the next thing
Speech to speech is way easier now at least, RVC kicks ass :) I'm going backwards and trying and struggling to learn how to do TTS ruclips.net/video/qZ12-Vm2ryc/видео.html&ab_channel=p3tro
Must haves gradio UI guide for voice clone training.
Thanks for these videos, they help a lot.
Appreciate it :D!
I've been toying with the idea of creating an AI assistant and have been dismayed at the amount of data I would need to send to multiple remote servers just to make it work. I found your video and "Dude! This can mimic any voice you train it on and it runs LOCALLY??? **Gasp** The possibilities..." The audio book functionality is particularly intriguing.
Id love a guide, im used to using Tortoise-TTS through cmd, but training voices using the version in your descriptions is hard for me but I think would make my voices better
Finished the latest tutorial on Tortoise-TTS GUI, let me know what you think! It's pretty fantastic!
2:55 I will be back for the setup video!
Thanks for all of this! As a writer, it's really useful for me to get good TTS audio readings for proofing. TTS will just read your typos straight, unlike me when I read my own work, so it's a cool tool to have. That this is available to make the voice listenable, unlike most TTS apps, and that it's free is really great.
Also, at 4:50, I'm on a 1070 and am able to get this to work. It's not even that slow. On a 1070, I was able to generate a reasonable-quality 20-minute reading in about 1-2 hours with your audiobook tool.
You need longer audio samples for Elevenlabs, 4 secs usually isn't long enough. You need about 10 seconds and with the slow pace of the speaker maybe even longer. Also you can do the same with changing and adding words.
Edit but EL doesn't have the training. I will have to look at the TTS.
Agreed, but even still, on longer samples, it messes up on the prosody (a fancy new word I learned lol).
Because you can "train" the tortoise models, it allows for more accuracy in this area which is what I've come to find. I also spent like 5k characters just trying to get a sentence sounding correct on Eleven Labs, which is definitely not cost efficient 😅😂
Now, I want to do it too! Thaks for your work!
Thanks Alfita, appreciate it!
Tortoise samples sounded more expressive but there is a weird vibrato in some cases that is very artificial...
Nah ngl I saw ur wallpaper and immediately had this massive sigh of relief and went okay this guys based
Whats the status on Non-english languages? Can I feed it samples of Chinese voice actor and then it will use that for English?
Sorry, but this does not even come close to 11 labs
Eleven Labs vs Tortoise TTS
You really went "nuh-uh"
It's supposed to be a meme
Not genuine discourse💀🤣
Hi Jarod, awesome video! I'm new to all this and im hoping to learn more about tortoise tts. It would be nice if you make a video help beginners install tortoise!!
bro i saw your comment is 6 months old. but I am new now. Tell me how to install and use it can you share anything that helped you the most
@@aryanhussain7583 I got it installed but I had some minor issues. I followed this youtube video for installation. ruclips.net/video/hOSsGOmDC3w/видео.html
I really liked the result, it even encouraged me to learn how to use it :))
I think the advantage of EL is the ability to generate in multiple languages.
I tested it in Portuguese (which is a bit complicated) and it worked very well
Thanks!
Indeed, it's much more straightforward using EL and the results are still really awesome. Looking right now to see how to use different languages with Tortoise TTS so hopefully I can get that working.
Portuguese is my country language and I will test too.
That's more than acceptable audio quality and honestly verges on the outstanding.
I agree, and I've been playing around with new techniques that further it even more :O
It’s good but as you hear it’s little bit robotic and idk how we can fix it 😅
Tortoise TTS is very solid! The only downside is the models eat up your HD space quickly, with each snapshot clocking in at 3 gigs!
Yuuuppp, which is why I really only save the latest model and delete all of the checkpoints etc. Eats up about 2GB per model for me
@@Jarods_Journey how do you delete the checkpoints?
4:49 I am curious how much GPU VRAM is consumed during a generation.
You can check by monitoring Task Manager -> Performance -> GPU memory
100% ish, depends. I've clipped by audio samples to be maxed by the voice chunks option, which I set to 0. If not, then you have to increase the voice chunks in order to chunk the audio
@@Jarods_Journey I use a python script I made that shows me the amount of gigabytes in use, so I can gauge "Oh this stable diffusion thing I can still edit videos while it runs"
~or~
"I have room to increase the batch size maybe"
Smartttt. I usually do one or the other though, if I'm training a model, that means now I get time to chill instead of editing a video haha.
1:20 Eleven labs is smoother / less glitchy sounding but also less emoted
Yup yup, it produces a much more "refined" voice, however, it's a little too clean. It loses the emotion and emphasis, as well as any dialect you might feed it.
Thy strength befits a crown!
Can't wait
Tortoise sounds so much better~
RELLY APPRECIATED WONDER FULL WORK.
so if your not a big coder etc etc... how do you install this simply
That honda accord part 🤣🤣
"Have you heard of the high elves?"
Cool Rem wallpaper yo
I'm the biggest fan lol :D
Thanks for sharing this with us. Please explain to us why all of those ai software do not come with an exe files do can be double click and it work like other portable or non portable software that it can be run locally on the computer without a need for internet or graphic card. Many of us do not have the knowledge of programing to make it having an exe file that can be double clicked and it runs locally on the computer..
nice to know i might "cheat" and "clone" voices from eleven labs lol
Hi...is there any way to control de speed of the generated audio? thanks
Do you have a video tutorial for setting up Local LLM? I played around briefly with Kobold and didn't get very far. I've had success with a bunch of things but local LLM and TTS are proving challenging.
What would use suggest using to create AUDIOBOOKS? every system seems to have very limited output lengths.
Will it work on other languages?
Very cool stuff! Do you have a link to the AI Audio Book Generator? Is there a way I could give the generator a full novel in a .TXT file and it automatically generates a .WAV or .MP3 of the full novel without having to run the sentences one at a time? Thanks!
Yup! I am working on a release for this so hopefully when I find the time, I can both upload and post a RUclips video about it
Good video!
hey i wanna that repo of bookmaker
Hello help me error
(When start Training get an Error:
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x84 in position 0: invalid start byte)
If you use the learning rate scheduler (calling scheduler.step()) before the optimizer’s update (calling optimizer.step()), this will skip the first value of the learning rate schedule. If you are unable to reproduce results after upgrading to PyTorch 1.1.0, please check if you are calling scheduler.step() at the wrong time.
Where is the how to actually make the model to run? I have models but how the hell do you get to this step. There’s no set up video still.
Skip to 17:00 of this video here: ruclips.net/video/6sTsqSQYIzs/видео.html&lc=UgwYIku4QJPhO6nPvgR4AaABAg
@@Jarods_Journey thanks. Do you have the link for the google colab to start the gradio program as mine only has 3 tabs I can’t seem to find the way to get the tts part to work. Only clones my voice or a aCapella. I can’t seem to tts. It always just says error or this is a sample , hope you like my voice model. If you have the colab link for your one on this video to load my models in that would be amazing. Great work, just wish I was a tech genius as u It’s annoying
Is Tortoise supporting multiple languages?
Awesome video! Quick question: If I my goal was to create music (singing) with AI trained voices, could I use Tortoise TTL for that? And, second quick question lol, is this best achieved natively with a PC or could I use my non-M1 MacbookPro? Thanks again for the great video! 🙏🏾
Singing is completely different architectures ATM, so you have to use SVC for singing, and for the computer question, it really depends on your systems specs! I believe it has to be Nvidia due to cuda for the speed, but I can't verify for other systems 😅
@@Jarods_JourneyThank you soooo much!
RVC is recommended
Ocupo la voz de ranni contando el lore de elden ring :D
I don't quite understand how this experiment works when Eleven Labs provides so many adjustable parameters.
It doesn't. You only have 2 parameters on EL and both are garbage tbh.
This repository gives you access to a lot of parameters that are hidden from user facing interfaces. Makes it much more customizable, which is how you can narrow it down to be better than EL imo
Is it possible to train my voice into this and give a script to it snd export the scripted voice as mp3?
hi, can you create tutorial how to make TTS GUI installation? thank you
i have models that i downloaded from huggingface meant for RVC do they not work on Tortoise TTS? if they do have you done a video tutorial on how to utilize them. thanks
Really impressive, even more so when the two were talking to each other! Any chance you make your trained models downloadable? Btw. I also got a few of those delete-me-folders. :-D
Appreciate it! Can't share the models unfortunately, but I will be sharing on how you can train your own model. The deleteme directory is becoming its own directory by now lmao, but at least I know I won't feel bad if I delete it 😂
@@Jarods_Journey It was worth a try, I mean "Traveller from beyond the fog. I am Melina." That's just EPIC and I'd like to have an epic assistent. ;-) Though I'm afraid that there's no way yet to plug Tortoise TTS into Oobabooga or Silly Tavern. I'd really like to dive into scripting AI stuff but my hamster wheel keeps me too busy and there is sooo much to learn. So aside from some minor fixes in the Stable Diffusion batch processing I couldn't do much so far. Btw. my delete-me folders are actually the successor of my various temp folders because everytime I intended to do some clean-up I'd already forgotten if there was something important in them and I'm somewhat of a data messy. ;)
i want you to help me i need to clone my voice to use it in excel tutorial videos to speak english well which github you do you recommend
what is this video player ?
Anyway, that tech is insane. I imagine modder being have to create custom quests in game fully voiced or indie game dev.
Yeah, 100% crazy. The video player is mpv: mpv.io/
How many languages Tortoise tool support ? I live in South Earth Asia
Thanks for the video. The instructions were very clear and it was easy to set up. Despite training and adjusting the settings I don't think it's quite as accurate as 11Labs, maybe 75% there. Have you shared the code for the audiobook reader?
Clarity wise, 11L all the way as it's much more crisp. But for prosody... Tortoise TTS does a much better job at this (though less clear)
I'll get to uploading the audiobook reader to GitHub today, thanks for the reminder 😅
@@Jarods_Journey Thanks Jarods! But I can't seem to find the audiobook reader in your repo?
Please tell me, I will generate a Speech from the text, can I use it for commercial purposes?
Hi Jarod, for the two AIs talking to each other, how do you get them to have no latency when responding to each other? When I use the 11 labs API, there is a few seconds of silence while the audio is being generated, then the AI starts talking.
Since everything is local, back when I did this my GPU was fast enough to keep up once the audio queue was full, meaning the audio was being generated faster than it could be played. It's gotta be done in some type of thread or queue so that the code isn't being blocked while audio is being read
Any chance you have tutorials on how to install tortoise on W10?
Hey what happens if we run this on superior hardware? I am into publishing and want to make audiobooks. So I need hyperfast. Dont mind spending a bit on compute if the end product is cheaper than 11labs. Does it speed up on paid colab plans?
It does indeed speed up on superior hardware, but by how much I can't give you concrete numbers on. You might wanna check out my video on the 3060 vs 4090, but the 4090 only inferences closer to 2x faster than the 3060 which is different than the related training speeds (4-5x).
Beyond that comparison, I'm not sure where the bottleneck in the system is for faster speeds as I haven't dived that deep into it yet.
Hey there! Does it work for multilingual??
lost my sh*t on Honda Accord my man!!
Is there a way I can get the voices other people trained?
What is the inference speed? I know there are wild variables - but to use with a local chat llm using a 4090, is it anywhere near real time?
Depends on the size of the LLM, but you'll be getting 5-10 seconds of delay on speech with tortoise
i've been searching for an option to train a model for tts
And this is it :)! A tutorial on its usage should be coming later this week
Do you know if it's possible to switch to different languages, like German? Thanks for the video!
It is, but takes a little bit of tokenization knowledge. I have no clue how to do that but you might wanna check out this channel here: youtube.com/@nanonomad
I believe he does train in German
Dude, can you write in more detail how to install it? what do you need to download? for more details please, I'm completely ignorant, I see you have a visual code and how to install it, what language to choose there, etc. please for more details
More details granted: ruclips.net/video/6sTsqSQYIzs/видео.html
SMH I can't keep a straight face for "finger maidens" If I played that game I would make up NSFW lore as the game progressed
LOL well I'm sure you're not alone in that, probably entire reddit groups dedicated to Elden Ring Alt. Lore lol
@@Jarods_Journey LOL I do that when I'm playing games or watching movies when I see something weird or funny
what about others languages ??
Great video! I want to use Tortoise TTS instead of Eleven Labs because I don't want to pay. Could you help me implement or tell me how to implement Tortoise TTS in the AI assistant script? It will be very helpful. I am eagerly waiting for new videos. Have a nice day.
A video for that is in the pipeline, if you wanna see more you could try watching some of my Livestreams until I get the chance to talk about it more in a video
If I am unable to complete the task on my own, I will use your video as a last resort to succeed.
How much do you charge me for cloning a voice in Spanish?
Unfortunately, Spanish would require a bit of work for tokenization andddd I don't sell my services atm. I can point you towards the right direction though and this guy seems to have done tome of tokenization for other languages youtube.com/@nanonomad
Does it work with diffrent languages such as arabic, germen. Etc?
It can train, but you need custom tokenizers
Hey man! Thanks for great guide! I have a question if you or someone know how to train TTS on other language. I want to clone my voice in czech so TTS will be also in czech. Can you point me right direction? Thanks a bunch!
You'll need a tokenizer and you'll have to train the model to understand a new language. Might wanna check out another guy called nano nomad who has done this
Any fixes for "CUDA out of memory" issue?
CUDA Out of Memory Issue
any other alternatives with better interfaces?
Yes xttsv2
Better than RVC? Its my question.
They're completely different architectures and software unfortunately, so the comparison is nearly impossible to make. For text to speech, definitely my favorite that I've come across so far.
I have an idea for a new video. Well, since I'm an absolute dunce, could you make a video explaining all this ai voice stuff, maybe perhaps a separate one for ai image stuff as well.
Like, I know squat about coding, and most of these things seem to require some level of coding to truly understand, with all the errors and GitHub non sense. Maybe you could explain how GitHub or similar sites work since whenever I go to one of those sites, I dont see a big glowing shiny button that says "DOWNLOAD" so I'm instantly lost. And when I finally download something, there are never instructions to learn how to actually set up the program, like with this one, I downloaded it but can not figure out how to start it up on my own.
Perhaps you can also explain how you learn and know how to do these stuff, which to you, seem like second nature, but to dummies like me are like solving calculus equations. Maybe also explain how to make the voice training sets for these voice ai's too? I don't really know how to go about doing something like that, if I need a specific file to store them in or something else.
The step-by-steps on my channel will come, they just take many more hours to make than the showcase ones (which is this one). I didn't show it here, but the author of the Git.ecker page was friendly enough to give us a step-by-step guide to follow: git.ecker.tech/mrq/ai-voice-cloning/wiki/Installation. You better bet I asked plenty of questions to ChatGPT about what many of those things meant.
We all start from somewhere! In the beginning, everything was quite grueling and rough, it took a lot of time to get used to these things but the hard work I put in has seemingly paid off! With the free availability of ChatGPT, I really recommend you just ask it anything and everything that is confusing, and then ask it to simplify things even more. I like to ask it "Explain it to me like I'm a 5 year old" and this usually helps me get the gist of topics quicker than normal so that I can have a relative understanding for how it works. Haven't gotten around to my workflow for learning new things yet, but I plan to so that others might be able to follow in suit!
@@Jarods_Journey Thank you so much for the feedback. Yea, I somehow forget how useful chatgpt is so thank you for reminding me. It really helped me out when I was struggling in my classes so I should've thought of using it in this situation lol.
@@hughsilva5655 Absolutely, the tool is a godsend. Helps so much with literally EVERYTHING 😂
Any ia like this to sing?
Not bad video, but for a video about audio, your own audio is too sharp, kinda hurts my ears, (pop filter, noise gate or compressor should help)
its work indonesian voice sir?
It would need an indoesian tokenizer for a different language other than english. A bit beyond what I've tried out so far.
Can we train a hindi language model in it.
It's possible, but you need a custom tokenizer. Check out the issues area on the gitecker page or go check out nanonomad on youtube.
@@Jarods_Journey thanks mate. I'll try.
Better then eleven labs ? How ?
If you're looking for matching prosody, Tortoise TTS does a much better job. If you're looking for clarity, well Eleven Labs get the job done. I've spent more than 50k character tokens in Eleven labs just to get a voice sounding "right" so that is quite expensive tbh.
I use tortoise a lot and your voices sound really bad, why not turn up the iterations? You will get a much better audio quality that way
Bad :o, in what way?! Where is this sorcery for better settings 😂?
No amount of iterations is gonna save my 2 minutes of training data 😂, in my testing, iterations above 100 don't add much to the voices. On a case of 10 samples each, 32 & 64 sounded better than 200 and 512. As well, the higher I go, the more prosody I seem to lose
This seems to be more a dataset issue than settings imo. Crisp data samples are better than noisy samples, more is better, etc etc. For my use case processing for an additional 4-5 seconds is too long as I need to get it as close to realtime as possible
Noice
You have audio content, but you pollute it with bad background music. Awful.
Free?
Yep
Clean audio huh? I'm here trying to work with Shodan lol
Lol, cleaner the better xD. If not, it'll mimic the "noise" or reverb.
I'm just curious if it will preserve the glitches and noise in her voice
Is it completely free?
100%
Better than EL? are you joking me?
If you are talking about intonation, sure, but Tortoise sounds like shat
Prosody wise, Tortoise is better by a long shot. EL misses out a lot there. Clarity wise? EL is much higher than Tortoise TTS. My bank account wise? I think Tortoise takes the cake 😂.
@@Jarods_Journey EL can pass as a real voice, Tortoise cannot.
@@korbpwBuddy, EL is BASED off of Tortoise. They literally forked the project and made their own model for it... Not going to go into the technicalities, but the ONLY reason Tortoise sounds "bad", is because the author of the project willingly tuned it down to avoid misuse thereof. Want to learn more? Use Google... It's not that hard.
@@realtyrone Bruh, let me make my own project worse.
@@korbpw Well, he did. Maybe he was paid by EL.
Tortoise is terrible, why would you actually recommend this
Free and quality is just fine ruclips.net/user/shortsKkm8ICjFG-k
@@Jarods_Journey I've yet to see anything beat eleven labs, they are the best on the market right now until their source code gets leaked