This is exactly what I was looking for the past one month, and suddenly popped up on my recommendation! Can't thank you enough for this. You saved my semester!!
1 minute into the video, absolutely amazed by the high high quality of this video. You are my favorite programming youtuber along with FireShip and NomadCoders! Thanks so much Nicholas!
Thanks so much for all these great tutorials! I’ve discovered your channel a few days ago, your way of teaching makes it really easy to understand and learn. I was wondering if you’d be able to do a series or video around recommender systems: building a recommendation engine (content-based, collaborative filtering), rather Netflix (movie) recommendations, Spotify’s music recommendation (could include audio modeling) or Amazon (purchases) predictions. Many thanks! Keep up the amazing tutorials :)
Can we use same if I wanted to seperate my bpos call recording from a conversation files, like if I train it based on my bpos recording and after that if I give a audio will it seperate my bpos sound?? Please help
I worked on similar project on Audio classification hooked up on raspberry with some sensors three months ago but using rcnn and librosa. A different approach from yours basically the same steps. Thumb up mate!!
how can i implement detection for example when the initial capuchin call started and ended how can i get this information in the end result (like in result it should show capuchin call - 2.3s)
Nice tutorial. A few inaccuracies there though about stft() usage: "abs()" there is not for getting rid of negatives, but for complex values amplitude. frame_length would probably better be power of 2...
how can i implement detection for example when the initial capuchin call started and ended how can i get this information in the end result( like in result it should show capuchin call - 2.3s)
The classification works on sliding windows of fixed size (3sec in this tutorial). One can slide the window with overlap to try to approximate the start of the matching sequence, or use other signal processing methods to find start of the sequence.
good tutorial. Note that sample rate has nothing to do with the amplitude of an audio file but rather the number of times the audio file is sampled per seconds.
if anyone get error at 22:14 for pos = tf.data.Dataset.list_files(POS+'\*.wav') neg = tf.data.Dataset.list_files(NEG+'\*.wav') just use the / instead of \. pos = tf.data.Dataset.list_files(POS+'/*.wav') neg = tf.data.Dataset.list_files(NEG+'/*.wav')
Thank you so much for these nice tutorials! They are quite helpful! I have a small question. I saw your process of building up models and training and testing them. If I want to spend less time in classifying the model, do you think it's possible to introduce some existing datasets such as esc-10 or esc-50 in your method?
Thanks for such a great tutorial. I have a question: What happens when resampling is done to an audio file? Does its total time changes or its number of sample changes or both changes or it depends on specific algorithm?
Us humans can hear up to 22kHz approximately and due to Nyquist frequency, you need to sample it twice as its higher frequency, hence that 44100Hz you may have seen. However, on audio analysis, most useful data is found in up to 8000Hz, so we resample it up to 16000Hz, losing the rest of higher freq. The length of audio does not change. What changes is the amount of bits we need to save the audio.
Another great video. I love the setup and style of your videos now, have to love a good whiteboard! I'd be interested in seeing a drone video - I've recently got a Ryze tello. Enjoying the python package to control, and computer vision capabilities that come with it!
LET'S GO, NICK :D This is actually pretty awesome and once again one of those things that i feel you can do TONS of different things with. Voice commands and animal calls are obvious examples, but maybe you could build a device that listen for a human's breath and heartrate and stuff like that and detects irregularities? This could be used for diagnostic purposes, but also as a warning device for for example elderly people. The moment it hears weird heart or breath sounds or something, it gives a warning and tells them to see a doctor. Same thing can be applied in a bunch of different fields, i think. Listening for weird engine sounds for example to help diagnose engine problems before internal parts suddenly and violently become external parts. Also, astronomy! Listening for gravitational wave events and things like that, though I'm pretty sure they're already using tons of AI for this anyway, so it's probably being done already. By the by, you posted about crowdsourcing labels/labeled data the other day, i think that's a great idea, especially if you're sharing that labeled data with the public! Doing it this way is a much more manageable way to get labeled data since hopefully the work is being done in parallel by distributed resources ("many humans") and sharing it online means that this project essentially helps everyone wanting to play around with ML and use the data for something cool, learn about things and maybe come up with awesome ideas to change our future. So, awesome idea and great way to leverage the outreach you have here on YT, I love it! ❤ What i really liked about this video in particular is the exploratory data analysis and the preprocessing alongside it. As we all know, it's very important to feed your data to your network in a way that makes it as easy to digest as possible, so learning more about this is absolutely essential and really fun aswell! Much appreciated! As always, Nick, thanks a ton for your videos and for doing all this for us, much much appreciated! Really love this video and looking forward to the next one! All the best, have a great week! :)
Heyyy, I had a good laugh at this comment "before internal parts suddenly and violently become external parts" 😂 but yes definitely agree! I'm hoping the crowdsourced labelling can become a thing! I know there's datasets out there but for a lot of the niche and practical stuff I have in mind, I can't really seem to find anything. I figure if people are willing to help label then I can give back to the community by showing everyone how to use and build with it! Have an awesome weekend!!
Hey nicholas, this project is great but how do i classify multiple animal calls using this model? I can classify the audio as capuchin or not capuchin this way but if i included more audio classes, how could i classify the audio file as the animal as well as the number of counts ?
Amazing tutorial. Really enjoyed that Nick 🙏🏼 I guess in case we wanted to detect multiple sounds from the same animal the procedure would be the same but we would need an equal number of samples to train the neural network? Furthermore in case we wanted to detect sounds from multiple animals and categorize them would we follow the same procedure just at the end we would put softmax instead of sigmoid ?
If you're encountering a shape issue when calling model.fit and getting the error "cannot take length of shape with unknown rank," the solution might be to explicitly set the shape of your tensors during preprocessing.
Amazing job Nicholas!! I have just a question, why didn't you calculate also the standard deviation of files' lenght so to have a more precise interval for your window?
Thank you for the great comment. How do you classify multiple sounds in a file and get time information as well like a sound was made at time 5 seconds into the audio file and another was made at 8 seconds how do we get time and class?
Thanks for uploading a great video and one question! You sliced an audio file from the very start of the file. I think the position where we start slicing can affect the model accuracy. For instance, if you skip 1s and start slicing, it may yield different wav data. Do you think slicing audio from the very start of the file is golden rule?
Great video thanks so much, I have a small question, In the line Spectogram = tf.signal.sftf Why you choose Fram_step =320 Fram_length=32 Can some one explain the method of choosing this please?
how can i implement detection for example when the initial capuchin call started and ended how can i get this information in the end result ( like in result it should show capuchin call - 2.3s)
data = data.map(preprocess) in this part im getting an error -----------------> TypeError: tf__decode_wav() got an unexpected keyword argument 'rate_in' although rate_in is the parameter of tfio.audio.resample
Always given inspiration of new knowledge...You are Great Thank Advice please Please my question is that can I buy Graphic card of 2G(NVidia for starting Computer Vision because my PC is too slow when I'm using my CPU..
Awesome explanation. Please help me to create audio augmentation for health records without losing information. I worked with time_shift(-0.5 to 0.5 variation in the wav). But, model ACC is not up to the mark.
Is it possible to run this while i have my mic always listening and to do live proccesing on that? Btw this will be my first project and i know its a lot.
I subscribed to this channel last October But today I see that is unsubscribed today I feel it's better to list all channels in a piece of paper and keep it in wallet
58:00 - Another great one, thank you, already looking forward to applying. Quick question - why mix the audio signals on the MP3. I get that it gets us to one channel - is there a way to just process one channel at a time. Im imagining that would allow for some spatial awareness in the model? Or perhaps too many variables because we are just looking for the one sound? Thinking that it would be useful to associate density with directionality... but not sure that's accurate if the original recordings were not setup to actually be directional...
I think this might also introduce distortion in the result, since we have to deal with stereo microphone separation, ideally about 100-150mm for human perception. I think the best idea is to just look at one channel in case of stereo, at least if the microphone separation is high or unknown.
Im not sure about the batch and prefetch part, for me i generates empty training sets afterwards and also it only takes 8 prefetched files for training?
Hello Nicholas, I have been using this great video in my beginners courses and last year everything was fine. Unfortunately in today's lecture the code did not run on any of my machines or configurations ... The load_16k_ wav function is not able to resample the audio files, and much worse, the model.fit function complains about an input that could not be -1, 0 or 1. Are you willing to check and update your code ? (Spend 6 hours now to find the error😊) Thx jtm
Tried to recreate the same. Somehow the program abruptly stops at hist = model.fit(train, epochs=4, validation_data=test) It has to do with using lot of RAM. Anyway for me? Thanks!
thanks for the great tutorial as always sir in case i want to make voice recognition to identify for which person is this voice is this steps will help me to reach that or do i need to look for something which is more specific to this task.
What happens when there are multiple birds in the dataset. Now how good is a CNN model on this kinda dataset? Again the source training audio samples can vary in length, sometimes in minutes. How can we do the pre processing in such cases?
i am getting a error and tried to debug so many way but not solve. Error is "The procedure entry point could not be located in the dynamic link library - to Positive and Negati a', 'Parsed_Capuchinbird_Clips') a', 'Parsed_Not_Capuchinbird_Clips') C:\ProgramData\anaconda3\lib\site-packages\tensorflow_io\ python\ops\libtensorflow_i0.50."
Hello Nicholas, thanks for you video :) I have a question I am aiming to build a model for sound detection using deep learning algorithms ( I am thinking about using LSTM). for now I am done with preprocessing step. I have the spectrograms of the sounds (generated using Short time Fourier transform) also I have the labels (binary labels as arrays, 0s where there are no events and 1s where the events are present). I am now confused about who to fed this data to the model. The shape of each spectrogram is (257, 626) and the shape of each label is (626,). How should I give this data to the LSTM. Can I build a model that takes the spectrograms with their current shape and give the labels as sequence of ones and zeros or I have to segment the spectrograms and give each segment a label?
If I want to count how many times 1 specific dog barks / day ? Then it is clear that samples of this dog barking are needed, but how many? And what other sounds must be sampled and how many if at the same place many other sounds can be heard? Thx!
Heya, I would suggest starting out with 100+ samples of the dog, you can then augment with different background sounds and white noise to build out the dataset. This is purely a starting point though, would need to do error analysis from there to determine where to put emphasis next!
@@NicholasRenotte It would be very interesting if an accurate counter can be made. This is a case of animal abuse ( dog in a small cache and neglected by the owner and barking for attention for over 2 years), so an accurate counter would be very helpful and usefull for other projects. What I not understand is why 1 sample/spectogram of the barking dog would not work good enough to detect it in a recording of for example 24h, because there must be very few or no sounds that have the same spectogram. I understand it will always be different but can 2 different sounds ( cat and dog for example) have 2 spectograms that are very similar? So my question is why it is not possible to identify a specific sound in a recording by comparing the spectogram of the sound to detect to all possible spectograms in the recording ms after ms? If you accept payed projects I would love your help, because this is all new for me. Regards!
@@SA-oj3bo I think you could. If there were multiple dogs in the sample you would probably need a ton of data though to be able to clearly identify which dog is barking. This would allow the net to pick up the nuances for that particular bark.
I've gotten through to the part where you start testing the predictions and my validation_data isn't showing up. The epochs run fine, but I have no val_precision or val_loss values. All I have are loss and precision_2. Any fixes?
23:30 What is the significance of that len(pos) or len(neg)? When len(pos) is replaced with 2 , I expect only first 2 sample data to have '1' label. However when I run -> positives.as_numpy_iterator().next(), I get '1' labelled not only for the first 2 samples but also for the rest.
Hi i am working with a new audio dataset which does not require audio slicing part? What should I modify to loop through the folder for the last part. Any help would be greatly appreciated
Hi Mr. Nicholas, I thank you for your RUclips content, I benefited a lot from it I am a PhD student in electronics. I am programming a robot. I am working on a program that is similar to the video, real time sign language. I am facing some errors and I could not get past them. If I can send a picture of the error and correct it for me, please
@@Maxwell-fm8jf in # Restore checkpoint ckpt = tf.compat.v2.train.Checkpoint(model=detection_model) ValueError: `Checkpoint` was expecting a trackable object (an object derived from `TrackableBase`), got . If you believe this object should be trackable (i.e. it is part of the TensorFlow Python API and manages state), please open an issue.
Dear Mr. Renotte, I'm trying to use this tutorial as a basis to build a classifier over several music genres. The only thing that I don't know how to adapt is the last layer of the CNN. How should I modify it so that it can get me as output let's say 10 different labels ? Should the labeling be modified upstream ? (I want to have 10 outputs instead of 1 at the last Dense layer, but I can't just modify it like it, so I'm wondering how I should do it) Thanks a lot !
Since the time you asked the question, you have probably solved it. However, my guess is that for your multi class problem, you should first have data for the 10 classes (samples of each of 9 music genres, plus a non classified genre) and the last layer should be a Dense layer with 10 neurons activated by a softmax function (instead of sigmoid) that gives the predicted probability of each class. You also have to change the loss function to be one of the 'categorical crossentropy' available in tf keras.
@@armelayimdji Hey Armel Tjanks a lot for the advice ! Obviously I'm done with my project and I went for something else : Instead of taking the spectrograms as input of my CNN, I extracted features from the sound wave and all the physical aspects of the music to have an input vector of features that I passed through an MLP and it worked well !
hey nick, i was rebuilding this project of yours now on a local machine - i was struggling with the spectogram size and memory allocation for my gpu rtx3070.. i downscaled to frame_length=160, frame_step=64 with input_shape=(748, 129,1) so far so good, now regarding the .fit of the model my training time is 50ms per epoch, final results tend to yours...did the colab train on cpu for the sake of memory with "highscale" spectograms maybe? could you run that locally and give some info? im very insecure regarding the train time of 50ms to the 3mins in the vid.. thanks anyways for the great tutorial man!!!
Hey great video, this is actually the one video I understood regarding audio classification. Also I have a question, I'm working on a musical instruments classifier which predicts what instrument/instruments is/are used in a wav file. So please can you give me some suggestions or the changes I have implement in my project as I'm referring to your video throughout in the development of my project. It Would be a great help
@@Joker-lr5he hello... what is the progress now.. im about to create a project which is more related to urs.. and then i just saw ur comment and popped up
This is exactly what I was looking for the past one month, and suddenly popped up on my recommendation!
Can't thank you enough for this. You saved my semester!!
1 minute into the video, absolutely amazed by the high high quality of this video. You are my favorite programming youtuber along with FireShip and NomadCoders! Thanks so much Nicholas!
Thanks so much for all these great tutorials! I’ve discovered your channel a few days ago, your way of teaching makes it really easy to understand and learn. I was wondering if you’d be able to do a series or video around recommender systems: building a recommendation engine (content-based, collaborative filtering), rather Netflix (movie) recommendations, Spotify’s music recommendation (could include audio modeling) or Amazon (purchases) predictions. Many thanks! Keep up the amazing tutorials :)
Definitely! I’m doing my own little deep learning challenge atm, will add it to the list!
Can we use same if I wanted to seperate my bpos call recording from a conversation files, like if I train it based on my bpos recording and after that if I give a audio will it seperate my bpos sound?? Please help
@@NicholasRenotte please answer my question, I really need some direction.🙏🙏🙏
41 minutes into the video. Not even a second I was bored. Amazing
I worked on similar project on Audio classification hooked up on raspberry with some sensors three months ago but using rcnn and librosa. A different approach from yours basically the same steps. Thumb up mate!!
Woahhh, nice! What was the latency like on the rpi? Noticed when I started throwing more hardcore stuff at it, it kinda struggled a little.
What you have developed in your project?
What where you using the pi for and have any tips on how to make a system that recognizes certain sound in real time?
Another great content from Nick. Thrilled to do a project on this
Yess! Let me know how you go with it!!
You’re the absolute best with these tutorials
Nice one!
Looking forward to a multi label text classification if you can!
Thanks
Yup, code is ready, should be out this week or next!
@@NicholasRenotte yo thanks a lot!
The way you get inputs from the community and interact is nice to see
A much needed tutorial !! btw can you make a tutorial on tts synthesis? not with pyttsx3... train a model to speak from provided voice data of a human
You got it!
@@NicholasRenotte 😃
Mc Shubap is spinning the decks in your memory palace 😆 Great tutorial so far.
Just discovered this channel on my recommended. Really awesome stuff man! Thanks for the great content.
This video is Awesome!!!
I got to know from this video that we convert Audio data to image data, to approach audio related tasks in ML!!!
Thank you so much for these tutorials, Nicolas. Will be great a tutorial about few shot learning. Grettings from México!
Awesome sound classification project👍I need a capuchino break after hearing the capuchind bird sound😎
Great video. Would love to see a version of this done using the GPU, with multiple classifications, not just binary.
Tensorflow has GPU builds
Super, super helpful getting my project off the ground!
Man these tutorials are amazing. Congrats for creating such great content. And thank!!
16:36
44.1KHz is the sampling rate. Meaning every 1/44.1K sec, the amplitude of the audio wave is sampled.
how can i implement detection for example when the initial capuchin call started and ended how can i get this information in the end result (like in result it should show capuchin call - 2.3s)
bro you are absolute gold, thank you so much for all the effort you put on your videos and teachings
+1 subscriber
Nice tutorial. A few inaccuracies there though about stft() usage: "abs()" there is not for getting rid of negatives, but for complex values amplitude. frame_length would probably better be power of 2...
how can i implement detection for example when the initial capuchin call started and ended how can i get this information in the end result( like in result it should show capuchin call - 2.3s)
The classification works on sliding windows of fixed size (3sec in this tutorial). One can slide the window with overlap to try to approximate the start of the matching sequence, or use other signal processing methods to find start of the sequence.
Thank you very much for your hard work. I really enjoy the video.
Amazing content and explanations. You have a new subscriber and fan!
Welcome to the team @Steve, glad you liked it!
good tutorial. Note that sample rate has nothing to do with the amplitude of an audio file but rather the number of times the audio file is sampled per seconds.
Incredible video, much appreciated, on the side note, I love your face cam, also audio is excellent!
Bro, great video and very good detailed explanation. 👍👍👍
It's him, he is back.
Ayyyyy Uriel!! What's happening!! Thanks a mill!
@@NicholasRenotte Always faithful man, while there deep learning tutorials we will be there
Awesome, kindly make a video on Speech Recognition model using Deep Learning.
you are wanderfull sir
if anyone get error at 22:14 for
pos = tf.data.Dataset.list_files(POS+'\*.wav')
neg = tf.data.Dataset.list_files(NEG+'\*.wav')
just use the / instead of \.
pos = tf.data.Dataset.list_files(POS+'/*.wav')
neg = tf.data.Dataset.list_files(NEG+'/*.wav')
thank you
What a great video is this. Thank you so much !!! 😍
What an amazing video. Definitely earned my sub.
Very Cool video Nick Bro!!
Thanks homie! Good to see you @Ronak!
Thank you so much for these nice tutorials! They are quite helpful! I have a small question. I saw your process of building up models and training and testing them. If I want to spend less time in classifying the model, do you think it's possible to introduce some existing datasets such as esc-10 or esc-50 in your method?
Very smart! can you identify a person speaking in the midst of others speaking more quietly?
Great content, keep it up man!
can I use this for classifying different guitar chords' audio?
Thanks for such a great tutorial. I have a question:
What happens when resampling is done to an audio file? Does its total time changes or its number of sample changes or both changes or it depends on specific algorithm?
Us humans can hear up to 22kHz approximately and due to Nyquist frequency, you need to sample it twice as its higher frequency, hence that 44100Hz you may have seen. However, on audio analysis, most useful data is found in up to 8000Hz, so we resample it up to 16000Hz, losing the rest of higher freq. The length of audio does not change. What changes is the amount of bits we need to save the audio.
Another great video. I love the setup and style of your videos now, have to love a good whiteboard! I'd be interested in seeing a drone video - I've recently got a Ryze tello. Enjoying the python package to control, and computer vision capabilities that come with it!
Definitely!! I've got one floating around somewhere, will get cracking on it! Thanks for checking it out @Anthony!
LET'S GO, NICK :D
This is actually pretty awesome and once again one of those things that i feel you can do TONS of different things with.
Voice commands and animal calls are obvious examples, but maybe you could build a device that listen for a human's breath and heartrate and stuff like that and detects irregularities? This could be used for diagnostic purposes, but also as a warning device for for example elderly people. The moment it hears weird heart or breath sounds or something, it gives a warning and tells them to see a doctor.
Same thing can be applied in a bunch of different fields, i think. Listening for weird engine sounds for example to help diagnose engine problems before internal parts suddenly and violently become external parts.
Also, astronomy! Listening for gravitational wave events and things like that, though I'm pretty sure they're already using tons of AI for this anyway, so it's probably being done already.
By the by, you posted about crowdsourcing labels/labeled data the other day, i think that's a great idea, especially if you're sharing that labeled data with the public!
Doing it this way is a much more manageable way to get labeled data since hopefully the work is being done in parallel by distributed resources ("many humans") and sharing it online means that this project essentially helps everyone wanting to play around with ML and use the data for something cool, learn about things and maybe come up with awesome ideas to change our future.
So, awesome idea and great way to leverage the outreach you have here on YT, I love it! ❤
What i really liked about this video in particular is the exploratory data analysis and the preprocessing alongside it.
As we all know, it's very important to feed your data to your network in a way that makes it as easy to digest as possible, so learning more about this is absolutely essential and really fun aswell! Much appreciated!
As always, Nick, thanks a ton for your videos and for doing all this for us, much much appreciated!
Really love this video and looking forward to the next one!
All the best, have a great week! :)
Heyyy, I had a good laugh at this comment "before internal parts suddenly and violently become external parts" 😂 but yes definitely agree!
I'm hoping the crowdsourced labelling can become a thing! I know there's datasets out there but for a lot of the niche and practical stuff I have in mind, I can't really seem to find anything. I figure if people are willing to help label then I can give back to the community by showing everyone how to use and build with it!
Have an awesome weekend!!
Thank you so much!!
Great tutorial! Thanks!!
This is amazing
Thanks for a great video. Could please explain why we need tf.abs in preprocess function?
Hey nicholas, this project is great but how do i classify multiple animal calls using this model? I can classify the audio as capuchin or not capuchin this way but if i included more audio classes, how could i classify the audio file as the animal as well as the number of counts ?
Amazing tutorial. Really enjoyed that Nick 🙏🏼
I guess in case we wanted to detect multiple sounds from the same animal the procedure would be the same but we would need an equal number of samples to train the neural network?
Furthermore in case we wanted to detect sounds from multiple animals and categorize them would we follow the same procedure just at the end we would put softmax instead of sigmoid ?
It's a very interesting video. But can we do the test using sound sensor?
Thanks so much sir, one request sir can you explain about how to classify audio as positive, negative and neutral
Amazing job
If you're encountering a shape issue when calling model.fit and getting the error "cannot take length of shape with unknown rank," the solution might be to explicitly set the shape of your tensors during preprocessing.
can you elaborate further ?
Amazing job Nicholas!! I have just a question, why didn't you calculate also the standard deviation of files' lenght so to have a more precise interval for your window?
Thank you
Eran
VERY COOL!!
Thank you for the great comment. How do you classify multiple sounds in a file and get time information as well like a sound was made at time 5 seconds into the audio file and another was made at 8 seconds how do we get time and class?
This was awesome 🔥
Thanks for uploading a great video and one question! You sliced an audio file from the very start of the file. I think the position where we start slicing can affect the model accuracy. For instance, if you skip 1s and start slicing, it may yield different wav data. Do you think slicing audio from the very start of the file is golden rule?
How to decide the values of frame_length and frame_step in tf.signal.stft(wav, frame_length=320, frame_step=32) ? Appreciate any help !
Maybe a basic question, but what does zero padding do when getting the frequency spectrum?
Great video thanks so much,
I have a small question, In the line
Spectogram = tf.signal.sftf
Why you choose
Fram_step =320
Fram_length=32
Can some one explain the method of choosing this please?
how can i implement detection for example when the initial capuchin call started and ended how can i get this information in the end result ( like in result it should show capuchin call - 2.3s)
Grazie!
Hello sir!
I installed tensor flow as presented but I don't understand the reason of the error message,no module named tensor flow.
1k! ) //THANK YOU! 🤗
Please make video on image segmentation (semantic and instance).
Coming soon!
data = data.map(preprocess)
in this part im getting an error -----------------> TypeError: tf__decode_wav() got an unexpected keyword argument 'rate_in' although rate_in is the parameter of tfio.audio.resample
Always given inspiration of new knowledge...You are Great Thank
Advice please
Please my question is that can I buy Graphic card of 2G(NVidia for starting Computer Vision because my PC is too slow when I'm using my CPU..
I would be looking at something with more RAM is you can swing it @Benya, 2gb is a little too small for most image computer vision tasks.
Use Google Colab’s TPU (Tensor Processing Unit).
It’s very fast if your model use only CNN.
@@NicholasRenotte Thanks 🙏
Awesome explanation. Please help me to create audio augmentation for health records without losing information. I worked with time_shift(-0.5 to 0.5 variation in the wav). But, model ACC is not up to the mark.
Had a lot of issues getting this to work. You need python 3.8.18 for this to work. I had that version of python on my conda env.
@NicholasRenotte The kernel appears to have died, It will restart automatically. Got this error while running model.fit. How can I solve this?
Is it possible to run this while i have my mic always listening and to do live proccesing on that? Btw this will be my first project and i know its a lot.
I subscribed to this channel last October
But today I see that is unsubscribed today
I feel it's better to list all channels in a piece of paper and keep it in wallet
58:00 - Another great one, thank you, already looking forward to applying. Quick question - why mix the audio signals on the MP3. I get that it gets us to one channel - is there a way to just process one channel at a time. Im imagining that would allow for some spatial awareness in the model? Or perhaps too many variables because we are just looking for the one sound? Thinking that it would be useful to associate density with directionality... but not sure that's accurate if the original recordings were not setup to actually be directional...
I think this might also introduce distortion in the result, since we have to deal with stereo microphone separation, ideally about 100-150mm for human perception. I think the best idea is to just look at one channel in case of stereo, at least if the microphone separation is high or unknown.
Im not sure about the batch and prefetch part, for me i generates empty training sets afterwards and also it only takes 8 prefetched files for training?
Hello Nicholas,
I have been using this great video in my beginners courses and last year everything was fine. Unfortunately in today's lecture the code did not run on any of my machines or configurations ... The load_16k_ wav function is not able to resample the audio files, and much worse, the model.fit function complains about an input that could not be -1, 0 or 1.
Are you willing to check and update your code ? (Spend 6 hours now to find the error😊)
Thx jtm
Tried to recreate the same. Somehow the program abruptly stops at
hist = model.fit(train, epochs=4, validation_data=test)
It has to do with using lot of RAM. Anyway for me? Thanks!
did you solve this issue?
thanks for the great video, is it possible to use .mp3 files in python instead of .wav to save disk space?
what approach is the best to find a number of blows in the recording of a Jackhammer?
thanks for the great tutorial as always sir
in case i want to make voice recognition to identify for which person is this voice
is this steps will help me to reach that or do i need to look for something which is more specific to this task.
what computer specifications do you have? training the model for me took 8 hrs
Hey Nicholas, I was wondering if there was a way to change the code to print the timestamps in the audio clip that each sound is recognized?
What happens when there are multiple birds in the dataset. Now how good is a CNN model on this kinda dataset? Again the source training audio samples can vary in length, sometimes in minutes. How can we do the pre processing in such cases?
i am getting a error and tried to debug so many way but not solve. Error is "The procedure entry point could not be located in the dynamic link library
- to Positive and Negati
a', 'Parsed_Capuchinbird_Clips') a', 'Parsed_Not_Capuchinbird_Clips')
C:\ProgramData\anaconda3\lib\site-packages\tensorflow_io\ python\ops\libtensorflow_i0.50."
Hi Nick ! Thank you for your extremely useful contribution to the AI learners' community
Thanks a lot
Hello Nicholas, thanks for you video :)
I have a question
I am aiming to build a model for sound detection using deep learning algorithms ( I am thinking about using LSTM). for now I am done with preprocessing step. I have the spectrograms of the sounds (generated using Short time Fourier transform) also I have the labels (binary labels as arrays, 0s where there are no events and 1s where the events are present). I am now confused about who to fed this data to the model. The shape of each spectrogram is (257, 626) and the shape of each label is (626,). How should I give this data to the LSTM. Can I build a model that takes the spectrograms with their current shape and give the labels as sequence of ones and zeros or I have to segment the spectrograms and give each segment a label?
If I want to count how many times 1 specific dog barks / day ? Then it is clear that samples of this dog barking are needed, but how many? And what other sounds must be sampled and how many if at the same place many other sounds can be heard? Thx!
Heya, I would suggest starting out with 100+ samples of the dog, you can then augment with different background sounds and white noise to build out the dataset. This is purely a starting point though, would need to do error analysis from there to determine where to put emphasis next!
@@NicholasRenotte It would be very interesting if an accurate counter can be made. This is a case of animal abuse ( dog in a small cache and neglected by the owner and barking for attention for over 2 years), so an accurate counter would be very helpful and usefull for other projects.
What I not understand is why 1 sample/spectogram of the barking dog would not work good enough to detect it in a recording of for example 24h, because there must be very few or no sounds that have the same spectogram. I understand it will always be different but can 2 different sounds ( cat and dog for example) have 2 spectograms that are very similar? So my question is why it is not possible to identify a specific sound in a recording by comparing the spectogram of the sound to detect to all possible spectograms in the recording ms after ms? If you accept payed projects I would love your help, because this is all new for me. Regards!
@@SA-oj3bo I think you could. If there were multiple dogs in the sample you would probably need a ton of data though to be able to clearly identify which dog is barking. This would allow the net to pick up the nuances for that particular bark.
I've gotten through to the part where you start testing the predictions and my validation_data isn't showing up. The epochs run fine, but I have no val_precision or val_loss values. All I have are loss and precision_2.
Any fixes?
23:30 What is the significance of that len(pos) or len(neg)?
When len(pos) is replaced with 2 , I expect only first 2 sample data to have '1' label.
However when I run -> positives.as_numpy_iterator().next(), I get '1' labelled not only for the first 2 samples but also for the rest.
Hi i am working with a new audio dataset which does not require audio slicing part? What should I modify to loop through the folder for the last part. Any help would be greatly appreciated
AMAZING
Thanks
hey nicholas,trying to execute this but facing issue as you havent done any preprocessing on the training data,looking for help from you
Please make video on making very own speech recognition using tensor flow
Why are we reducing the sample rate from 44khz to 16khz, it was not clear to me?
Hi Mr. Nicholas, I thank you for your RUclips content, I benefited a lot from it I am a PhD student in electronics. I am programming a robot. I am working on a program that is similar to the video, real time sign language. I am facing some errors and I could not get past them. If I can send a picture of the error and correct it for me, please
What error did you get? I worked on similar project but different approach on rpi hooked with some sensors.
@@Maxwell-fm8jf in # Restore checkpoint
ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
ValueError: `Checkpoint` was expecting a trackable object (an object derived from `TrackableBase`), got . If you believe this object should be trackable (i.e. it is part of the TensorFlow Python API and manages state), please open an issue.
How did you uploaded the file as colab does not support folder upload until it is in zip file format
Great video Nich 👏, can you do a tutorial about sign language to text convert which takes sign videos not sign images as dataset, I am stuck with it 😴
Definitely, added video classification to the deep learning list yesterday @Masia!
@Nicholas Renotte thanks
Dear Mr. Renotte,
I'm trying to use this tutorial as a basis to build a classifier over several music genres. The only thing that I don't know how to adapt is the last layer of the CNN. How should I modify it so that it can get me as output let's say 10 different labels ? Should the labeling be modified upstream ?
(I want to have 10 outputs instead of 1 at the last Dense layer, but I can't just modify it like it, so I'm wondering how I should do it)
Thanks a lot !
Since the time you asked the question, you have probably solved it. However, my guess is that for your multi class problem, you should first have data for the 10 classes (samples of each of 9 music genres, plus a non classified genre) and the last layer should be a Dense layer with 10 neurons activated by a softmax function (instead of sigmoid) that gives the predicted probability of each class. You also have to change the loss function to be one of the 'categorical crossentropy' available in tf keras.
@@armelayimdji Hey Armel Tjanks a lot for the advice ! Obviously I'm done with my project and I went for something else : Instead of taking the spectrograms as input of my CNN, I extracted features from the sound wave and all the physical aspects of the music to have an input vector of features that I passed through an MLP and it worked well !
I have similar kind of problem, can you help me?
hey nick, i was rebuilding this project of yours now on a local machine - i was struggling with the spectogram size and memory allocation for my gpu rtx3070.. i downscaled to frame_length=160, frame_step=64 with input_shape=(748, 129,1) so far so good, now regarding the .fit of the model my training time is 50ms per epoch, final results tend to yours...did the colab train on cpu for the sake of memory with "highscale" spectograms maybe? could you run that locally and give some info? im very insecure regarding the train time of 50ms to the 3mins in the vid.. thanks anyways for the great tutorial man!!!
hello, can I ask you about this things?
Hey great video, this is actually the one video I understood regarding audio classification. Also I have a question, I'm working on a musical instruments classifier which predicts what instrument/instruments is/are used in a wav file. So please can you give me some suggestions or the changes I have implement in my project as I'm referring to your video throughout in the development of my project. It Would be a great help
Also, will it be a better idea to go by the mfcc approach?
@@Joker-lr5he hello...
what is the progress now..
im about to create a project which is more related to urs..
and then i just saw ur comment and popped up
please help me, why doesn't colaboratory open this code? How should it be changed so that it can be opened in the colaboratory?