Title says "How to convert PDFs to audiobooks with machine learning" it does not say "How to convert PDFs to audiobooks for the first time ever groundbreaking discovery with machine learning". Appreciate the work and explanation.
A lot of people don't understand the sense of fulfilment and satisfaction that comes with building things yourself. Using existing apps is a good idea but building your own opens you up to a whole world of new possibilities. Thanks for sharing!
@@dalemarkowitz8021 guess someone needs to build a machine learning model/project to classify the parts of a page into different categories then we decide what part we read based on what "we know" it is
@@moustafarahal3396 I am thinking like training a TensorFlow objection detection model with labeled pages to detect "important" portions and then feed the OCR with cropped portions... IDK if it would be a super noob approach... (since i am :3 )
@@tusharmaurya1668it would be looking at the actual font size of the garbage text, not just the font size of the body, that way, the headings won't be excluded.
@@lvishal8556 isn't the problem how to identify the garbage text from the actual content? So, if we knew which one is the garbage text which we would have to know to give a different font as you said, we wouldn't have to go through all this trouble
I achieved this myself about 12 years ago but not using ML. I merely formatted the PDF files so that they were easily processed by the voice engine in Acrobat Reader. It worked fine although I didn't have the range of voices you have now. I got used to 'Marvin' (sounded a bit like Hawkings) and listened to many books. I did go so far as to create .wav files from them but the data was massive, although converting to MP3 would have been possible. Still a great deal of data though. I had all of the text in a Word file which I would print to a Postscript file and convert to PDF using Ghostscript. This approach would have been useful but I think using a voice engine is a lot more efficient.
Why not do it the easier way? 1. Convert pdf to epub file. 2. Upload the epub file to google playbooks and then download it on the same app. 3. Use the read aloud feature and enjoy.
@@alexjr977 After steps1 and 2 . When u open the epub file and touch any page, u will find three vertical dots at the upper right corner. There u'll find the read aloud feature.
@@alexjr977 Also I was thinking why work so hard when things are easy? 😅 It's like u have a bike and u move around ur entire city to go to ur school which is just 15 minutes walk from ur home😂😂
This is a fantastic idea for any PDF, not just research papers. I would take it two steps further. 1) Optionally converting it into a podcast format. As a podcast, I can listen to it in Airr, which allows you to bookmark audio clips and transcribe those clips. This is like taking notes while you listen, and is long-walk friendly. 2) Put this up on a web front-end for public use.
that sounds like something I'd like. Often when I listen to things while walking I hear something I want to take note of and have thought of some solution like this.
1 had no clue 2 watch the video 2nd time ive got 20% 3rd time ive got 40% and 4th time ive got 70% wow no more for today i need to clear my head and watch it tommorow .....thenk you very much ,,,
Wow..This is legit the practical use of ML in real life which millions of people want to do..converting them into audiobooks is very helpful for college students and as a project to do it would be very helpful..Thank you, GCP...Now I know why I had subscribed to you guys..Thanks again :-)
Most PDF readers already provide this feature of 'read out aloud'. I have been using this feature for more than 9 years. But, it reads the page numbers as well which isn't a big deal.
You are a clever girl, said the Doctor. Love your delivery style and pace. The associated graphics are great for following the process. Keep broadcasting!!
I used balabolka to extract the text from the PDFs, used find and replace to remove the garbage and used Amazon Polly to save the audio to an S3. Downloaded 32 hours audio from academic papers for my MBA class.
Wow before this video came out, I put together a rudimentary version of this with AWS Polly. Your font-size based approach to cutting out the junk is so good.
@@hectorprx My script it iterates through the entire text and at every word, it adds the word length to a characterCount variable. I use the characterCount variable to make sure it grabs 100k characters or less at a time and if a word pushes it past the limit, I drop the word and send the request, before starting the next chunk. There's no way to increase the limit as far as I can tell.
I use this app called Read Aloud for PC and eReader Prestigo for Mobile. The voice sounds pretty natural to me - cutomizable. To add to that the text can be edited to your liking. As a lawyer, helps reading those lenghty judgements & books on the go. I am addicted to them now, can't imagine my life without them. No more scared of heavy texts and books - just need the pdf.
Since I do not have so much expertise in coding, what I did was used ocr and aligned the content in a word file. Then used the microsoft immersive reader to read out the text for me.
I did the exact same thing 40 years algo, but instead a computer I used a tape recorder and myself as text to speech. And as a player and headphones I used a walkman.
In many cases you can't be sure which of the non-standard font size text is "junk text" without manually looking it over. I wouldn't simply and categorically think of all headings and subheadings as "junk text". These parts of the document weren't put there for nothing. Their purpose is to help organize the information being presented. Once you see or have the heading/subheading read, you'll know, especially if you got distracted, where in the document you're at. You can also know, through the headings what's about to be covered. I'm also not really bothered about page numbers read out loud. To me that's kind of like an "audible progress bar". I don't mind knowing how far along I am in the audiobook converted pdf, lol. If it's a matter of speed and efficiency, increase the read out loud speed to 1.5x - 2x or whatever you can handle.
Yup. That's how it's done. Talk to someone who has done it, copy the code and tada you have built machine learning model that converts pdf to audio. I used to overthink about programing but this is a way to go.
i made a similiar thing where i used my camera to detect text and it will speak it to me word by word while also highlighting the word. It can work in realtime where you use your camera and it can also work in normal mode.
This is very cool! Though I fear the usefulness of reading academic texts aloud is limited because of the many figures and formulas they often contain. I do like the approach of choosing the most-used font-size as a heuristic indicator - nice thinking :)
I believe there is the voice 'sampler' method from the early 90's that has now come full circle and one's 'own' voice is convertible as are many celebrity voices.....Could you listen to yourself reading aloud to?....yourself?.......
I was thinking about a related question: what are the audio qualities of the things we hear everyday? Specifically to your idea: what are the audio qualities of a given voice that make it sound robotic vs natural? If an AI model can Van-Goghify a photo with "style transfer" then surely another model can apply a realistic mask or filter of sorts to make a generic robotic voice sound human. Ugh. Actually that's a hard problem.
There are apps that let you use tts on PDFs, it's a whole lot of more practical than using an mp3 file except that I haven't found a software that gives you the option to make the tts ignore certain texts(like page numbers, repetitive sentences at the top-bottom of texts etc.). So other than converting the PDFs to mp3 it would be more beneficial and productive to add advanced settings such as making tts ignore "garbage texts". If you guys develop a PDF reader like that, I'd definitely use it instead of the current app I am using
Have you ever heard about Narrator's Voice? It does the same thing, I'm using it since the last 3 years for making my college notes into audiobook, basically MP3 files.
Hehheh I am doing this since last 6 months by just copy pasting the text to text to speech Google platform. Although this is the smart and easy way now ;) thanks for sharing...
I use an app named "eReader Prestigio" which uses google text to speech engine allows me to listen like an audiobook. it's works actually better than I expected. and also they have paid version which uses their own developed text to speech engine which works similarly as shown in the video.
@@dalemarkowitz8021 I was clueless about how to do it. I'm interested on a using data set like InfluxDB or Prometheus. Would be grateful if you can initiate a video on that. By the way your videos are very good. Keep up the good work!!!
The masks, especially when worn appropriately, greatly reduces the spread of water droplets that come out of our our mouths and nose. Which the germs that gets transmitted to one another reside in
Been thinking of how to replicate Adobe Reader voice assistant in other documents which are open on other applications & now this GCP ML project popped up. Mega Cool
@@dalemarkowitz8021 I used "IVONA Voices 2" with "Adobe Reader" 's 'Rread Out Loud' option but "IVONA" was for PC only and paid we need something same with Android handset but free. Can you please suggest us about that !?
Omg! I was looking for such a solution eagerly! I can’t stand reading long texts for my studies! Please share how you coded it! Is there also maybe a way to convert scanned/photographed texts from books to audio? And: can I make it run on my iPhone (I guess not)?
I know we're all tech savvy and trying to create things, but the average guys out there just wants an app... which already exists as ebook/pdf readers with text2speach functions
Those pdf text2speech functions search for actual boxes containing selectable text. What this video shows is a process to recognize the text from a completely static PDF (almost like an image), THEN apply the speech2text functions.
I love the project, exactly wat I was looking for. However, I do (subjectively of course) prefer Azure's voice generator so that's what I'm using the final part for.
![Summer Walker - Heart Of A Woman [Official Lyric Video]](http://i.ytimg.com/vi/SFddctbZzw8/mqdefault.jpg)
Title says "How to convert PDFs to audiobooks with machine learning" it does not say "How to convert PDFs to audiobooks for the first time ever groundbreaking discovery with machine learning". Appreciate the work and explanation.
A lot of people don't understand the sense of fulfilment and satisfaction that comes with building things yourself. Using existing apps is a good idea but building your own opens you up to a whole world of new possibilities. Thanks for sharing!
lol, i am literally working on this exact project, with the exact APIs, to solve the exact problem of listening to technical books
Lol! I'm curious to know how you handle the problem of deciding what to include in the audio!
@@dalemarkowitz8021 that's autoML Tables task, I think
and i gonna build a startup from it hahahahaa
@@dalemarkowitz8021 guess someone needs to build a machine learning model/project to classify the parts of a page into different categories then we decide what part we read based on what "we know" it is
@@moustafarahal3396 I am thinking like training a TensorFlow objection detection model with labeled pages to detect "important" portions and then feed the OCR with cropped portions... IDK if it would be a super noob approach... (since i am :3 )
I thought there's an app ready to download here.
No, she just converted the pdfs into mp3 files.
Same here.😒
me too🤣
And my Moon+ reader can do this automatically without any machine learning 😄
@@ihtesham_emon thanks
Using that font size approach to remove garbage text was clever!
But it will also remove the headings and sub-headings (b/c it is not in common font size)....
U can give the garbage text a different type font not size. Then u can just remove that type font. In that way no other issue will be seen I guess
@@tusharmaurya1668it would be looking at the actual font size of the garbage text, not just the font size of the body, that way, the headings won't be excluded.
@@lvishal8556 isn't the problem how to identify the garbage text from the actual content? So, if we knew which one is the garbage text which we would have to know to give a different font as you said, we wouldn't have to go through all this trouble
@@archimidiz then what's ur approach ?
I achieved this myself about 12 years ago but not using ML. I merely formatted the PDF files so that they were easily processed by the voice engine in Acrobat Reader. It worked fine although I didn't have the range of voices you have now. I got used to 'Marvin' (sounded a bit like Hawkings) and listened to many books. I did go so far as to create .wav files from them but the data was massive, although converting to MP3 would have been possible. Still a great deal of data though. I had all of the text in a Word file which I would print to a Postscript file and convert to PDF using Ghostscript.
This approach would have been useful but I think using a voice engine is a lot more efficient.
I did same 5 years ago, using pyPDF and ESpeak!
Instead of MP3 convert to AAC
There's a lot of stuff I'm missing out just because I'm not resourceful enough huh
@@asandax6 how
The art of Bodge !
I use @ Voice Aloud Reader. It's simple and has many voices too.
Why not do it the easier way?
1. Convert pdf to epub file.
2. Upload the epub file to google playbooks and then download it on the same app.
3. Use the read aloud feature and enjoy.
boi . Where is read aloud
@@alexjr977 After steps1 and 2 . When u open the epub file and touch any page, u will find three vertical dots at the upper right corner. There u'll find the read aloud feature.
@@adiparzival4682 bro i upload this on website not in app .But Thank you for sharing knowledge 😀🤗
@@alexjr977 Also I was thinking why work so hard when things are easy? 😅
It's like u have a bike and u move around ur entire city to go to ur school which is just 15 minutes walk from ur home😂😂
@@adiparzival4682 Lol
Don't do with Mathematics books.
Hahahah
Why ull read maths i thought u solve math
This is funny!
Dead haha
It's okay, if you use it for math books, it will also be incomprehensible like if you read it yourself. So the neural net does its job well.
I am completely in love with this Google tool! The videos on my channel will be made completely like this and even in two languages!
This is a fantastic idea for any PDF, not just research papers. I would take it two steps further. 1) Optionally converting it into a podcast format. As a podcast, I can listen to it in Airr, which allows you to bookmark audio clips and transcribe those clips. This is like taking notes while you listen, and is long-walk friendly. 2) Put this up on a web front-end for public use.
How do you do this?
that sounds like something I'd like. Often when I listen to things while walking I hear something I want to take note of and have thought of some solution like this.
This type of project puts a smile on my face. It’s fun, useful, and simple to understand and implement
simple uh?
This is so exciting - perfect for my Dyslexic daughter to use at school - thank you!!!
You have shared the approach highlighting the ML functions & other GCP modules.
Gives a clear pathway in to how problem was resolved.
Good one.
Add this tech in an e-book reader and launch something like kindle, Google
yo HALO fan
It's already been there for years
Google Play Books and Kindle have a read aloud option. AND Microsoft Edge's pdf viewer has a read aloud option.
I am pretty sure Google books has that
There are apps like that but they don't have advanced settings over the TTS, a shame really
Some PDFs are actually images so it’s necessary first to convert the file into intelligible text and afterwards audio files.
1 had no clue 2 watch the video 2nd time ive got 20% 3rd time ive got 40% and 4th time ive got 70% wow no more for today i need to clear my head and watch it tommorow .....thenk you very much ,,,
Wow..This is legit the practical use of ML in real life which millions of people want to do..converting them into audiobooks is very helpful for college students and as a project to do it would be very helpful..Thank you, GCP...Now I know why I had subscribed to you guys..Thanks again :-)
Aw, glad to hear it! That's why we make 'em! :)
@@dalemarkowitz8021 is it thefree service
Most PDF readers already provide this feature of 'read out aloud'. I have been using this feature for more than 9 years.
But, it reads the page numbers as well which isn't a big deal.
What a cool concept, Thanks for bringing it alive!
You are a clever girl, said the Doctor. Love your delivery style and pace. The associated graphics are great for following the process. Keep broadcasting!!
I used balabolka to extract the text from the PDFs, used find and replace to remove the garbage and used Amazon Polly to save the audio to an S3. Downloaded 32 hours audio from academic papers for my MBA class.
Hello, I was using Polly also but limited to a 100 k of characters which required batching the job. Did you figure out a work around ? Thanks
@@hectorprx nope, let me know if you ever do
Wow before this video came out, I put together a rudimentary version of this with AWS Polly. Your font-size based approach to cutting out the junk is so good.
aws polly excellent tool
Hello, I was using Polly also but limited to a 100 k of characters which required batching the job. Did you figure out a work around ? Thanks
@@hectorprx I used polly way before I didn't have much idea at now
@@hectorprx My script it iterates through the entire text and at every word, it adds the word length to a characterCount variable. I use the characterCount variable to make sure it grabs 100k characters or less at a time and if a word pushes it past the limit, I drop the word and send the request, before starting the next chunk. There's no way to increase the limit as far as I can tell.
Great Video! But I was wondering if you could do a step-by-step video because I am having trouble with the minute details?
Google: Upload video
Also people: *There are many apps existing before you*
Well, GCP is for developers so the video uploaded is targeted for developers to build their own in Google's cloud
I use this app called Read Aloud for PC and eReader Prestigo for Mobile. The voice sounds pretty natural to me - cutomizable. To add to that the text can be edited to your liking. As a lawyer, helps reading those lenghty judgements & books on the go. I am addicted to them now, can't imagine my life without them. No more scared of heavy texts and books - just need the pdf.
Pocket app does this really well. I'm not sure if it handles PDFs but their own voice outputs are really good.
Finally... A channel without ads
The @Voice Aloud app for Android has been reading PDF research papers, articles, and entire books to my ears since 2015!
This is already available in adobe few months back.
I like the "garbage text"....I gives me a clue that something is changing, or there is something I need to actually look at.
I've been using text aloud for all my pdf. So far so good. It reads blogs too or whatever on the screen.
Since I do not have so much expertise in coding, what I did was used ocr and aligned the content in a word file. Then used the microsoft immersive reader to read out the text for me.
I am really amazed. I have a lot of books in pdf format.
And have gone through the same phase.
This was what .i was working on few years ago but approach used in this was amazing..
The amount of views in this videos show how much good this problem is to teach as an example to google's cloud products.
Bring this soon i need to study with this for my exams😜
I did the exact same thing 40 years algo, but instead a computer I used a tape recorder and myself as text to speech. And as a player and headphones I used a walkman.
In many cases you can't be sure which of the non-standard font size text is "junk text" without manually looking it over. I wouldn't simply and categorically think of all headings and subheadings as "junk text".
These parts of the document weren't put there for nothing. Their purpose is to help organize the information being presented. Once you see or have the heading/subheading read, you'll know, especially if you got distracted, where in the document you're at.
You can also know, through the headings what's about to be covered. I'm also not really bothered about page numbers read out loud. To me that's kind of like an "audible progress bar". I don't mind knowing how far along I am in the audiobook converted pdf, lol.
If it's a matter of speed and efficiency, increase the read out loud speed to 1.5x - 2x or whatever you can handle.
smart shortcut using the font-style hack...waaaay better than several days of marking-up for training.
I've been using balabolka and the old ivona voices to achieve these seamlessly. Glad to see this on GCP. I'll definitely check this out. Thanks
yup the good old Balabolka
Yup. That's how it's done. Talk to someone who has done it, copy the code and tada you have built machine learning model that converts pdf to audio. I used to overthink about programing but this is a way to go.
i made a similiar thing where i used my camera to detect text and it will speak it to me word by word while also highlighting the word. It can work in realtime where you use your camera and it can also work in normal mode.
I am trying with different approach, like removing the words (
Interesting & very useful concept for Google Android app making.
Wonderful and practical use of machine learning.
This is very cool! Though I fear the usefulness of reading academic texts aloud is limited because of the many figures and formulas they often contain.
I do like the approach of choosing the most-used font-size as a heuristic indicator - nice thinking :)
I suppose it highly depends in what academic field you are. For a lot of social sciences it could be quite nice.
*laughs in social sciences*
Pls launch such feature this year this is amazing
The next thing we need to focus on is how to convert text to speech in natural voice, rather then that robotic
I believe there is the voice 'sampler' method from the early 90's that has now come full circle and one's 'own' voice is convertible as are many celebrity voices.....Could you listen to yourself reading aloud to?....yourself?.......
I was thinking about a related question: what are the audio qualities of the things we hear everyday? Specifically to your idea: what are the audio qualities of a given voice that make it sound robotic vs natural?
If an AI model can Van-Goghify a photo with "style transfer" then surely another model can apply a realistic mask or filter of sorts to make a generic robotic voice sound human.
Ugh. Actually that's a hard problem.
can you do a step by step walk through of this procedure please? thanks
There are apps that let you use tts on PDFs, it's a whole lot of more practical than using an mp3 file except that I haven't found a software that gives you the option to make the tts ignore certain texts(like page numbers, repetitive sentences at the top-bottom of texts etc.). So other than converting the PDFs to mp3 it would be more beneficial and productive to add advanced settings such as making tts ignore "garbage texts". If you guys develop a PDF reader like that, I'd definitely use it instead of the current app I am using
try moon readers built in tts - you can modify settings to ignore stuff.
What about for those of us who can't buy the "bucket" on google?
Thank you Dr Markowitz and chalom!
Have you ever heard about Narrator's Voice? It does the same thing, I'm using it since the last 3 years for making my college notes into audiobook, basically MP3 files.
Useful for reading anything which primarily requires your critical thinking, a good aid indeed.
Hehheh I am doing this since last 6 months by just copy pasting the text to text to speech Google platform. Although this is the smart and easy way now ;) thanks for sharing...
Really informative and interesting. Looking forward to more videos based on projects on ML/AI
How about building an AI that reads text and summaries it? What extra steps will be needed apart from the above mentioned ones?
GPT3
Yeah exactly. GPT-3 is your answer.
There is an app.
eReader Prestige.
It reads pdfs like audiobooks. It has local languages too.
Removing garbage text was pretty clever though 😍
Please let the next project be: how to do a "literature review" using AI? this would be a revolutionary step in the research world.
This is an interesting idea!
I’m exploring this
Copy other similar topic's literature review and change the words. No plagiarism, no time spent and ready made references lol
literature review? you mean like sentiment analysis? Or summary?
the latter is far more interesting, like an ELI5 app.
@@zapy422 That's dope, best of luck!
Good job and very interesting! Thanks
neat project i might have to try to do it myself
Necessity is the mother of inventions
This is cool project. Is it expensive though? cool, I saw the answer now. Thanks!
Fantastic
Now give an option for procrastinators
This is really insightful. Thanks for sharing!
that is just awesome! we are living in the future! finally
You do know right that there is an app named Moonreader ?
Will there be an app for that, it would be really great to listen to paper while I'm on the treadmill.
Whoa!! Some part of the speech sounds like real human talking, specially the last part. . . . Great work. .
You should try J on -4 pitch.
I use an app named "eReader Prestigio" which uses google text to speech engine allows me to listen like an audiobook. it's works actually better than I expected. and also they have paid version which uses their own developed text to speech engine which works similarly as shown in the video.
Really fantastic project!
Impressive work 👍
We have powerful LLM models these day. If you were to write this today, would you leverage them to filter out the junk words?
this is incredible!!!
Amazing work
Hi Dale, I would like to see how to use GCP ML services to use time series data from things like metrics and predict future behavior patterns etc.
Hmm, good idea!
@@dalemarkowitz8021 I was clueless about how to do it. I'm interested on a using data set like InfluxDB or Prometheus. Would be grateful if you can initiate a video on that. By the way your videos are very good. Keep up the good work!!!
Great video!
Wow, wonderful tips and tricks... I went through that ordeal!
_"So now that I'm in quarantine, I go out on walks without a mask"_
The masks, especially when worn appropriately, greatly reduces the spread of water droplets that come out of our our mouths and nose. Which the germs that gets transmitted to one another reside in
Why would you walk outside with a mask on
This is awesome! Thanks for sharing
3:47 no comments on the "perfect" voice choice!!?
Yes, I wanted this.
This is incredible piece of stuff!
Like to see more series like this. :)
Lol. There is a app moon reader for many years
FINALLY SOMEONE 🤣
Exactly
But what if someone want to make their own, just to try hands on technology.. and gain some experience.. (btw, thanks for the app suggestion..)
But the "speak" feature is for pro version only..
Are cracks available
Wow!!! This is literally the starter that I needed for my project.
Perfect, just what I have been wanting! Thanks :-)
It still sounds like machine voice. Instead try IVONA text to speech engine for TTS which has very robust natural voice in HD quality.
Been thinking of how to replicate Adobe Reader voice assistant in other documents which are open on other applications & now this GCP ML project popped up.
Mega Cool
Great I will use
What is the total API and GCP cost for converting 1 pdf(10 pages) to an audiobook?
For me, it was free, because I stayed in the free tier! The cost varies depending on your usage. See the blog post for more details.
@@dalemarkowitz8021 I used "IVONA Voices 2" with "Adobe Reader" 's 'Rread Out Loud' option but "IVONA" was for PC only and paid we need something same with Android handset but free. Can you please suggest us about that !?
@@dalemarkowitz8021 And one friendly suggestion, you have 24 'subscribers' including me :D, you should upload.
Omg! I was looking for such a solution eagerly! I can’t stand reading long texts for my studies! Please share how you coded it! Is there also maybe a way to convert scanned/photographed texts from books to audio? And: can I make it run on my iPhone (I guess not)?
I know we're all tech savvy and trying to create things, but the average guys out there just wants an app... which already exists as ebook/pdf readers with text2speach functions
That is why "average guys" don't bring a society forward. It is the innovation that brings the most value.
Those pdf text2speech functions search for actual boxes containing selectable text. What this video shows is a process to recognize the text from a completely static PDF (almost like an image), THEN apply the speech2text functions.
@@1stSilence you're right💯
Thank you for sharing
It seems so complicated though I would give it it a try thank you.
3:43 Look at me Rick i am your narrator. Im Narratormorty!
Thank you for the great video! how can we download the audio book after completing the process? appreciate if you can share
sounds very good
I love the project, exactly wat I was looking for. However, I do (subjectively of course) prefer Azure's voice generator so that's what I'm using the final part for.
Great work!!!
that is a lot of chrome extensions
that lacks intelligence.
I don't know if that's a joke. But those are not chrome extensions. They are Google cloud services and you need to configure and pay to use.
damn I was starting to make everything from scratch , thank god you showed up!!!!
Need this because I’m too lazy to read my essays.