transcription and speaker identification OpenAI-Whisper and Pyannote [Python]

Hello guys, in this video I will how you how to transcribe and identify the speaker by using OpenAI Whisper, Pyannote and Pydub .
For Pyannote you must register on huggingface website to get the access token.
Support me by subscribing to my channel and leave a like.
Github repository for the source code :
github.com/Mas...
OpenAi github link :
github.com/ope...
Pyannote github link :
github.com/pya...
Pydub github link :
github.com/jia...
#openai
#openai_whisper
#pyannote
#pydub
#python
#speaker_identification
#transcription
#diarization

[@bhuvneshsaini93]

Please provide requirement.txt, else its really very hard to make it workable.

[@leoncezammit2502]

Im really struggling to get this working, would i be able you to send you my output log ?

[@enriqueleonmacias249]

Wow, the transcript takes like two times the duration of the file to process. I guess that this solution wouldn’t work to monitor hours of call recordings unless you use gpu servers.

[@masteringpython]

it is recomended to use cuda ( nvidia gpu ) for speed
cpu is very slow

[@sakibzaman7719]

is it working on any other language?

[@hrishikeshnamboothiri.v.n2195]

try to include its requirements.txt also...
Thanks

[@lawrencemedina5593]

conda activate open_chatting does not work on my computer. "EnvironmentNameNotFound: Could not find conda environment: open_chatting
You can list all discoverable environments with `conda info --envs`."

[@masteringpython]

install conda toolkit then create an environment called open_chatting by typing :
conda create --name open_chatting
after that install the libraries that i mentioned in the video then run the code

[@chungrandy780]

Is there a colab version?

[@ryanschwartz3340]

nice video. Is the repo hard-coded to your directory structure? when I tried to change it, it said the format wasn't recognized

[@masteringpython]

do you mean segment file ?

[@JasminePlows-r4y]

Thanks for the demo. I am getting the following error, even while using your audio.mp3 file:
end = int(millisec(j[3]))
return (int)((int(spl[0]) * 60 * 60 + int(spl[1]) * 60 + float(spl[2])) * 1000)
ValueError: invalid literal for int() with base 10: ''

[@JasminePlows-r4y]

@mamido mami Yes, I did that, still getting the same error

[@auflute]

same problem

[@lunarl-l1k]

same problem

[@jbatista2008]

From the error message and the code, it seems that the error is happening because the millisec function is trying to convert an empty string to an integer.
The millisec function splits a time string, given in the format "hh:mm:ss.sss", into hours, minutes, and seconds, and then converts these components to milliseconds.
Here is an example of the string being parsed:
['[', '00:00:00.998', '-->', '', '00:00:20.622]', 'G', 'SPEAKER_01']
When this loop runs, it returns an empty 'end' string:
for l in range(len(k)):
j = k[l].split(" ")
start = int(millisec(j[1]))
end = int(millisec(j[3]))
The array position you want for 'end' is 4, not 3. Plus, it has a ']' symbol, so it must be cleaned up:
for l in range(len(k)):
j = k[l].split(" ")
start = int(millisec(j[1].rstrip(']'))) # remove trailing ']'
end = int(millisec(j[4].rstrip(']'))) # remove trailing ']'

[@ThePikkutyyppi]

can i use this program to split speakers to their own files? or is this only for transcription

[@masteringpython]

read more about pyannote to see how to split speakers

[@ThePikkutyyppi]

@@masteringpython What? Where?

[@EhsanEslahchi]

does this model work on languages other than English?

[@masteringpython]

onely english

[@PaweDuzy]

@@masteringpython Only english? What is I change model = whisper.load_model("small.en") to "small"? Acording to Whisper github documentation.

[@Yacine_zaki_abderrazzak]

Thanks man, you deserve the best

[@Hirotodoroki]

trying to run this but getting File contains data in an unknown format. tried several files and tried a wav file too, but no luck

[@masteringpython]

I advise you to use python anaconda to create development environment .Then install whisper openai ,after installing this library run a simple test to check if everything works correctly .Then install pyannote library and also run a simple test ( read carefully the installation guides maybe you missed something while installing the library)

[@nadeembaig5943]

@Hirotodoroki were you able to resolve the error (File Contains data in Unknown Format)?

[@bootneck2222]

Great video. Thank you. Can the output be displayed on screen whilst it is processing?

[@WhiteShark010]

You have chance.

[@kmillanr]

no code in video

[@patoyrigoyen]

Does this need GPU?

[@masteringpython]

in this video i did not used GPU, but if you want to use it read the pyannote documentation

[@ghulamshabbir9532]

do this work offline ?

[@masteringpython]

yes

[@userrjlyj5760g]

ما شاء الله تبارك أخ محمد .... شكراً لك

[@ApparaoMulpuri-d6m]

Hi, Thanks for the Video. Need approach on how we can implement the solution with the large Audio with duration of 3 hours.

[@KamilKaczmarekSolutions]

chunks

[@KamilKaczmarekSolutions]

chunks and saving .txt from these chunks in files, add logic to see what chunks it already has (if you face error or sth, and you want to come back and don't have to start over, just continue where it left off)