@@faheemahmadofficial7701 we need someone who can dub in English, Hindi, Telugu. Do you know these languages? If yes then My budget is 120 USD only. Per month full time.
Hindi in thumbnails ultimate way to get clicks ...haha But team the biggest issue for me today is subtitles. Custom models for my speach for hindi to english
I think, the glossary CSV thing can also be configured for subtitles API too, for some level of customisation as per the frequently used words. BTW mai aapki videos dekhta hu, your content is really great 🔥
Someone actually did it, it was IBM or Google I dont remember but they built an AI that could replicate someone's voice by hearing it read a specific line once. They didn't make it publicly available though 😂 I think we all know why. They did it for research and publicly showed results. It was pretty good.
Some language have different sentence for different gender like the one you used (hindi) , it was considering you as a male speaker, so I think they should use voice frequency(pitch) to identify gender in such types of language.
I am using Dubverse to dub videos using AI. The speakers are much better than any AI speaker out there. THey have most Indian languages that no one is doing.
There’s already an extension that does this so creators don’t have to bother with it! Here’s how anyone can set it up ruclips.net/video/BeS2zKSA6w4/видео.html
I actually would watch videos with this kind of translation. The future use for this is just amazing! For people who are too young to read or can't read a whole new world opens with this
My suggestion is why not google take this more seriously and a put this in RUclips as an option. Then RUclipss content will universally be accepted by any country people. This way it will be good for countries with less content in native language and lower prominence of RUclips.
There are a lot of factors that can not be solved that easy, specially without human input. Also you have to think of two scenarios one where a person has to watch a video and one where a person wants to watch a video. Depending on the end usecase, clientsatisfaction and viewtime may significantly rise or fall.
Matching the tone and pitch of voice would add so much to the dubbing. I know that at least in English there are various options to choose from. Not sure about other languages though.
🤣 REKT. I just uploaded an entirely Hindi dub here, though there was 0 human intervention so no promises on the quality! What do you think? ruclips.net/video/0y8uLxTkB3k/видео.html
I don't understand why RUclips isn't doing this already, so much content could be made available in native languages. Lots of comments here are asking for RUclips to do this. Do it RUclips!
The work is really great and explained in quite simplicity... Making it Real time may be a way with some difficulties but it's parameters are nicley tuned👍 thank you for sharing..
Nice!!! This is amazing! Great job! Im working on creating a machine learning model that trains with previous dubbed episodes and dubs future episodes with each character unique voice.
My last attempts with similar titles turned out that it works as long as the sentences are simple, but as soon as more complex sentences come or something has a nickname/name according to some other thing, the sentences absolutely stop making sense because translator change word order.
That's my dream to have a similar option like changing subtitles to a different language. And, same for the voice, to change the original voice to a couple of different languages in RUclips.
I had that idea in the past.We can't control first two steps.But third step with proper investment can be improved significantly.There are some technologies which can extract individual voice characteristics and make text to speech based on these characteristics.
Combine this with ai voice cloning like lyrebird and you can speak any language in no time... That makes something fascinating and scary at the same time
Hi Dale, I was thinking about it for a month or so that why not videos can be translated in voice mode. I think you should proceed with this process and brings it over to RUclips :)
I have a question, why is not Venezuela in the countries list for making a Google Cloud Platform Account? 😔😔 Is there a video of how to use Google Cloud Platform? Billing, Credentials, Api key, auth, with a step by step?
imagine how cool it would be if in future we can watch netflix video from another language to our native language even if it is not available but just using subtitle and Realtime video dubbing using AI
Thats great. The video is well planned. We are working on the same project as a final Year project of our college. Now we are stuck in how we will synchroniz the lips and pitch of the voice. Your suggestions or any source will be highly appreciated ❤.
Looks great!!..Is there as way to clone/mimic certain voice? I did some research but ended up with a github repo. It is not an end to end system, there the cloned voice should be post-processed manually for some acceptable results.
Hey Dale, One thing I noticed. In different languages the speech tempo is generally different. Think this can be auto-adjusted in a future revision of this work?
I am really wishing this is able now : Any updates on google translation capabilities? Say if one had a dance course can it hear .see what you are saying and dubbing into multi languages and give subtitles to each new language it produces? Would be really great if this exist now with all new AI capabilities ? Also if the translation would be able to change if it’s a woman talking or man in the same course to their audience
Hey google :) Would it be possible to mixe that technique with generatives networks to have lips synch by altering the video feed? Like Károly Zsolnai-Fehér of two minutes paper, couple paper down the road I can see a pipeline for automated video translation close to VO feel.
Dale here: Right now I sort of took the original audio and made it quieter when I was overlaying the subs. It might be hard to tell from the samples. I'm sure there's a better way to do it.
When restricted to using commodity services (e.g. google APIs), this is all good info. However, results from state of the art research is way above the quality of commodity services.
Early on I contemplated using ML to build a model that matches actor's voices to machine-made voices from the Google Cloud Text-to-Speech API. I would build a vision model using AutoML Vision that compares the mel spectrogram of voice samples. But I ran out of time. You should try it!
When will you release speech to speech translation? You wrote about this technology in Google AI Blog in 2019. ai.googleblog.com/2019/05/introducing-translatotron-end-to-end.html?m=1
I need to translate my own You Tube channel from Spanish to portuguese. I don't know nothing about programming. Can you help me? Or I can do that even though I donn't know nothing about programming? Thank you very much. ❤👍 Excellent video. This is just what I need💪👍
Great idea! Wonder if this could be something the streaming services could use e.g. for children's material. It is getting problematic getting shows dubbed in Icelandic for kids and it is affecting their ability to learn their native language
Hey! I’ve been doing a similar thing but my only probleme is that I need to match the new voice to the video and my video are from 30 min to 2 hours. Any ideas how to make this faster? I used premiere to transcribe, then a translator and finally an AI voice called Speechello.
Hi, I work doing subtitles.... I have my own methodology wich achieved to make it very fast with otter.ai... How could I use an already curated working SRT to make a speech for dubbing ??
Why this kind of effort. You just need is translate app ( from Google ). You speak it and translate it. Click on the speaker button at the bottom. That's it... 😂
@@daleonai7448 Yes, it does, but it doesn't dub the video. Would like to eventually be able to get dubbed translations for videos with a similar voice.
Get $300 and start running workloads for free → goo.gle/39F12fQ
Make an app and everyone can dub vidoes
yeah ! give more
I can hire people for $100 for entire month. It'll sound more real.
@@GauravImon please hire me bro i need to earn to buy a laptop
@@faheemahmadofficial7701 we need someone who can dub in English, Hindi, Telugu. Do you know these languages? If yes then My budget is 120 USD only. Per month full time.
Hindi in thumbnails ultimate way to get clicks ...haha
But team the biggest issue for me today is subtitles. Custom models for my speach for hindi to english
I think, the glossary CSV thing can also be configured for subtitles API too, for some level of customisation as per the frequently used words.
BTW mai aapki videos dekhta hu, your content is really great 🔥
@Gareeb Scientists .
Bhai big fan following your channel for almost 2 years
@@KishanKumar-ul1ef thank you. yea need to look into it as well as others
my fav utuber 🤩🤩
Tbh, I came here after seeing Hindi text lol
Saw the thumbnail in Hindi
Clicked asap
Same
Yeah that's called clickbaiting
Same bro, same
@@TheWohnJick You are indian ?
@@jatinoham yeah
Next level: use ML to synthesize the translated speech in the original speaker's voice
One day combine all of these techniques flawlessly and w/o delay - Babel Fish here we come!
They already have that.
Edit: or have I misunderstood? Did you mean that specifically she should do it?
Someone actually did it, it was IBM or Google I dont remember but they built an AI that could replicate someone's voice by hearing it read a specific line once. They didn't make it publicly available though 😂 I think we all know why. They did it for research and publicly showed results. It was pretty good.
Also sync the lips..
Next level: the person is augmented using AI
Indians:
Emergency meeting
@@kg3217 😂
😂😂😂
@@SangeetaSingh-sq1dx 😆
Some language have different sentence for different gender like the one you used (hindi) , it was considering you as a male speaker, so I think they should use voice frequency(pitch) to identify gender in such types of language.
I am using Dubverse to dub videos using AI. The speakers are much better than any AI speaker out there. THey have most Indian languages that no one is doing.
I've been wanting to work on this for very long.
I'd like to thank you for the brilliant explanation.
Pretty interesting. Maybe RUclips can implement this on scale to translate audio.
that means MASS LAY OFFFs in FILM INDUSTRY
@@nw91 AI is not as good as Dubbing as of now. However, it might also become very affordable to get a low quality ai-dubbing.
There’s already an extension that does this so creators don’t have to bother with it! Here’s how anyone can set it up ruclips.net/video/BeS2zKSA6w4/видео.html
I actually would watch videos with this kind of translation.
The future use for this is just amazing!
For people who are too young to read or can't read a whole new world opens with this
Yeah that's what VAs are for, maybe hire them instead of making a robot lol
My suggestion is why not google take this more seriously and a put this in RUclips as an option. Then RUclipss content will universally be accepted by any country people. This way it will be good for countries with less content in native language and lower prominence of RUclips.
There are a lot of factors that can not be solved that easy, specially without human input. Also you have to think of two scenarios one where a person has to watch a video and one where a person wants to watch a video. Depending on the end usecase, clientsatisfaction and viewtime may significantly rise or fall.
Matching the tone and pitch of voice would add so much to the dubbing. I know that at least in English there are various options to choose from. Not sure about other languages though.
Foreigners: Target Indian Audience for ez views
Google : Write that down write that down
🤣 REKT. I just uploaded an entirely Hindi dub here, though there was 0 human intervention so no promises on the quality! What do you think? ruclips.net/video/0y8uLxTkB3k/видео.html
Thank u mam.
For me this is very interesting topic .
Plse make continusely to this type of short but knowledge full video🙏
I don't understand why RUclips isn't doing this already, so much content could be made available in native languages.
Lots of comments here are asking for RUclips to do this. Do it RUclips!
The work is really great and explained in quite simplicity... Making it Real time may be a way with some difficulties but it's parameters are nicley tuned👍 thank you for sharing..
Thanks a lot 😊
This is actually I am looking for ,I am going to code this
Yes, i would watch. This would open so much content to the wider world.
Nice!!! This is amazing! Great job!
Im working on creating a machine learning model that trains with previous dubbed episodes and dubs future episodes with each character unique voice.
Ok thanks you gave me a new project idea
A step in the right direction. Kudos to the engineering team.
My last attempts with similar titles turned out that it works as long as the sentences are simple, but as soon as more complex sentences come or something has a nickname/name according to some other thing, the sentences absolutely stop making sense because translator change word order.
That's my dream to have a similar option like changing subtitles to a different language. And, same for the voice, to change the original voice to a couple of different languages in RUclips.
Simply brilliant and awesome idea and a perfect solution! Thanks Dale.
cool! surprised youtube doesnt offer this already as they do with closed captions
This video just made me consider learning machine learning.
Now that is some amazing use of Cloud and AI. Congrats!!!
Yes! Thank you!
Brilliant! I like that workflow ..seems easy! thanks for sharing
BRILLIANT!
THANK YOU!
OP! now use ML to synthesize the translated speech in the original speaker's voice
I had that idea in the past.We can't control first two steps.But third step with proper investment can be improved significantly.There are some technologies which can extract individual voice characteristics and make text to speech based on these characteristics.
I really love the idea:) Thanks.
Combine this with ai voice cloning like lyrebird and you can speak any language in no time... That makes something fascinating and scary at the same time
Wow awesome video!
Integrate this into RUclips please.
Local to global in one button click would be a game change.
Well done. 🙏
they can do this but wont... it means mass media i film industry lay offs. because everyone would see info from the source. it means less control
Hi Dale, I was thinking about it for a month or so that why not videos can be translated in voice mode. I think you should proceed with this process and brings it over to RUclips :)
Haha, thank you, I think computer voices are controversial though!
@@daleonai7448 Dale - honestly that's not a problem when it comes to understand what the video is all about :)
it should be default button on all you tube videos/netflix /prime, it opens whole new universe to people. It is exhilarating even to imagine that!
I have a question, why is not Venezuela in the countries list for making a Google Cloud Platform Account? 😔😔
Is there a video of how to use Google Cloud Platform? Billing, Credentials, Api key, auth, with a step by step?
Really amazing will we see this technology on RUclips and when?
they have aur face, retina, gov has thumb impression id and now they will have our voice in every language
Nice video, thank you!
Wow, i liked this video su much 😄😄
Amazing and useful project 😁👍👍
I thought Of doing this and slacked will do now, thanks :)
Loved the Project ❤️❤️❤️
*Great Idea, Great Implementation*
imagine how cool it would be if in future we can watch netflix video from another language to our native language even if it is not available but just using subtitle and Realtime video dubbing using AI
I love ML and AI can you tell me some best resource to understand it better...
Thats great. The video is well planned. We are working on the same project as a final Year project of our college. Now we are stuck in how we will synchroniz the lips and pitch of the voice. Your suggestions or any source will be highly appreciated ❤.
Thank you so much for this super useful vid, what is the lastest on A1 Dubbing, any easier way now days ?
If you explain it in detail then it will be very greateful
Btw greatwork
this was really great
amazing work
Why RUclips not use AI and ML for dubbing vdos ??
It will help lots of people.
Of right to start coding this....🌟
Yes! Yes! Yes!
i'll watch definately.
this is so cool!
Amazing
Thank you! Cheers!
holy cow, i didn't realize this tech was already here in 2021
Looks great!!..Is there as way to clone/mimic certain voice?
I did some research but ended up with a github repo. It is not an end to end system, there the cloned voice should be post-processed manually for some acceptable results.
Hey Dale, One thing I noticed. In different languages the speech tempo is generally different. Think this can be auto-adjusted in a future revision of this work?
I am really wishing this is able now :
Any updates on google translation capabilities? Say if one had a dance course can it hear .see what you are saying and dubbing into multi languages and give subtitles to each new language it produces? Would be really great if this exist now with all new AI capabilities ? Also if the translation would be able to change if it’s a woman talking or man in the same course to their audience
Hey google :) Would it be possible to mixe that technique with generatives networks to have lips synch by altering the video feed? Like Károly Zsolnai-Fehér of two minutes paper, couple paper down the road I can see a pipeline for automated video translation close to VO feel.
Can we also embed the background noise to make it more... you know... more realistic?
But, without that too, it is amazing.
Dale here: Right now I sort of took the original audio and made it quieter when I was overlaying the subs. It might be hard to tell from the samples. I'm sure there's a better way to do it.
This is awesome.....🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥
When restricted to using commodity services (e.g. google APIs), this is all good info. However, results from state of the art research is way above the quality of commodity services.
It's true, you can do things like directly go from speech-to-speech. But gathering the dataset and training those massive models... seems hard!
Genius 🔥
I would love RUclips to put this function 😃
that's awesome
can we change the audio o/p?
Wow great...
Awesome 😎
Imma try configure ur approach & dub the undubbed animes. Any suggestions on keeping the actor voice & accent intact?
Early on I contemplated using ML to build a model that matches actor's voices to machine-made voices from the Google Cloud Text-to-Speech API. I would build a vision model using AutoML Vision that compares the mel spectrogram of voice samples. But I ran out of time. You should try it!
awesome, thank you
Can't wait to get these in a software and dub some sub only anime
Like mission impossible ghost protocol movie
If you could add speed control like speaking faster or slower then it might add more naturality in voice quality . 👍👍👍
Ohhh well done.... how can I learn to do this from beginning level..?
do you have a service to translate a youtube video from Turkish to English ?
When will you release speech to speech translation? You wrote about this technology in Google AI Blog in 2019. ai.googleblog.com/2019/05/introducing-translatotron-end-to-end.html?m=1
I'm waiting for this too 🥲
You mean speech to speech dubbing, right?
I need to translate my own You Tube channel from Spanish to portuguese. I don't know nothing about programming. Can you help me? Or I can do that even though I donn't know nothing about programming? Thank you very much. ❤👍 Excellent video. This is just what I need💪👍
Great idea! Wonder if this could be something the streaming services could use e.g. for children's material. It is getting problematic getting shows dubbed in Icelandic for kids and it is affecting their ability to learn their native language
Google engineers using Hindi for example. PROUD TO BE INDIAN
It might be great If this goes online to RUclips. .
Hey! I’ve been doing a similar thing but my only probleme is that I need to match the new voice to the video and my video are from 30 min to 2 hours. Any ideas how to make this faster? I used premiere to transcribe, then a translator and finally an AI voice called Speechello.
Hi, I work doing subtitles.... I have my own methodology wich achieved to make it very fast with otter.ai... How could I use an already curated working SRT to make a speech for dubbing ??
thanku for this video]
Most welcome 😊
Omg, thanks.
But how do u remove audio dubbing of a movie with AI?
She is using VS Code For coding
speech to text could be more human sounding, but that will be more resources on the cloud servers.
Listen to the robotic voice for a long time is painful. What if we train the AI Model to use the same vocal cords of Speaker for voice.
Love it
Why this kind of effort. You just need is translate app ( from Google ). You speak it and translate it. Click on the speaker button at the bottom. That's it... 😂
Aadi you are so smart
Great 👌
Can you please add this functionality to RUclips?
So, I know YT has automatic captioning built in, and I thought it maybe had translation??
@@daleonai7448 Yes, it does, but it doesn't dub the video. Would like to eventually be able to get dubbed translations for videos with a similar voice.
@@daleonai7448 We want to watch without keeping our eyes off the scene and focusing on captioning
Yoooo google give me heart and I'll giv 🍪
I'm working on similar kinds of things... but the end goal is to make a similar kind of tone, not a robotic tone...
hello... Do you do anime??? I just watched your video..