..Thank you and THANK you! :) I was looking for what you explained so well. You sir just saved me a bunch of time! I was looking for a way to pick out the 10 most prominent spectral features in a spectrogram without having to write a lot of code myself. pls Keep it up! :)
I have just found your signal processing series. Excellent. I have a question that I hope you can shed some light one. I use SciPy fft, fftfreq, and find_peaks, and have found that the amplitudes of the peaks found are hugely different (i.e. fft amplitude of a peak is 1200 and find_peaks has an amplitude of 0.65. Any insight would be appreciated. Thanks.
Can you do one where you find the peaks , global maxima and minima, when the data is non stationary, like in the case of stock prices? Thankyou for this video!
Foward looking is alright, look ahead is fine as my usecase is different, but identifying them is the goal and I find it difficult. I would appreciate your help. :)
First of all, I would like to thank you for your explanation.. I have some questions for this method. My experimental data has 1600 information about modal analysis ( frequencies Vs Magnitudes ) So, how can identify the most good peaks with this PPM? (I'm not good in python)
Hi Somaya! a couple questions to guide the answer. How are you defining “good peaks” tallest peaks? Best resolved? What is the nature of the data? Scipy has a lot of tools for what you’re generally describing but which specific combination of methods will depend on the nature of the data and the questions above. Happy to help more!
Hi , i have the signal FFT of time domain, which is divided in segment, i want to find the peaks od each segment before applying clustering method, but i want to take both the frequency and magnitude into consideration in clusters, what should i do, i can see in you are taking just the magnitude of each peak ..
@@CJP3 the raw data was just the energy imbalance (generation-load) in hourly interval, it was in time domain , i created segment before applying fft . i took FFT to convert to frequency domain , now i want to find peaks but considering both Frequency and Time . i am sorry for my English if it is still not clear , i can phrase it differently . thank you for the time
Interesting to see your walkthrough and thanks for sharing. I've used script.signal.find_peaks for locating the position of plotted data in scanned images from means across each axis. Ie a0 = np.mean(img_a, axis=0) & a1 = np.mean(img_a, axis=1). Noise is always a problem but I've had luck subtracting a smoothed version from the original.
Man, this video is just awesome, but i am really struggling with how do i transform my RAW data into excel files so i can find the peaks in python. I am a coding beginner so i am still learning the basics, and your vid really help me seeing the bigger picture!
@@CJP3 Yes it is a .RAW data, im using an LC/MS instrument to get the data. I've tried to download the MZconvert and MS-DIAL but none of them worked for me. How do you convert your files in your Mac? Thanks for the kindness, your the best!
Hi, you would need to have a metric for the S/N threshold and sufficient signal processing to clean up the data. For that reason EDA and a visual assessment of the data is a good start. Furthermore, you could ML techniques like DBSCAN to estimate peak threshold values but this would still likely require an iterative process.
@@CJP3 Thank you for your shift response. Visual assessment is out of conversation as we try to automate exactly this. I will try the solutions you proposed. Much appreciated
Howdy! Thank for watching the video. Does this video get at what you're thinking about: ruclips.net/video/wAmlfQqhCh0/видео.html If not I'll dig more into this idea :)
Is there a way to have python count the number of peaks, I'm working with a data set that may contain a couple hundred peaks to a thousand peaks, I'm working on counting interference fringe lines from a laser aimed at an Arduino photo diode detector and I don't want to have to count up the red dots manually.
Hi Connor! It is possible to build a script to have Python count peaks. If using scipy.find_peaks you can measure the line the peak list. Len(peak_list). However, I think based on your description you have signal - peaks and interference - peaks the same spectrum.
@CJP3 when I comment out the scatter plot to check the p1 array thats generated, theres nothing in the array. For the most part I've copied everything line for line except the renaming of the columns and file path
@@CJP3 I figured it out, i had to omit the values in the line 'p1=indexes(df.v.values....)' to 'p1=indexes(df.v...), where v is my voltage values i want to use as the y value reference in the scatterplot.
Howdy, I took a poll about whether to keep or drop the background music and the jury was split. So instead I looked at the analytics and retention is higher with background music. To compromise I don’t have it in all the videos. Sorry it’s distracting!
![Denoising Data with FFT [Python]](http://i.ytimg.com/vi/s2K1JfNR7Sc/mqdefault.jpg)
Amazing video ❤❤❤
Thanks SpongeBob! 😂
I've watched several Python and data science tutorials lately. As somebody who doesn't really code much, this is EASILY the clearest one. Nice work.
Thank you very much! I checked out your roulette video 🔥 🔥 🔥
Good comment
Bro, I'm so proud of you! Thank you for this content :D
Thanks boss! 🙏🏽
A beautiful voice that sounds reminiscent of listening to a jazz singer.
🎵 that’s awesome!!! Thanks for the kind words 🎷!!
Thank you so much! Currently trying to work with analysis of absorption spectra graphs in uni, and that's shown to be really helpful!
You're very welcome!
Would appreciate more if no background music. Otherwise very interesting and well presented.
Thanks! The more recent videos have either less music or no music.
..Thank you and THANK you! :) I was looking for what you explained so well. You sir just saved me a bunch of time! I was looking for a way to pick out the 10 most prominent spectral features in a spectrogram without having to write a lot of code myself. pls Keep it up! :)
Glad it was helpful!! Thanks for letting me know!! 😃
Thanks so much, this was really helpful for me as someone new to coding! I've used this to analyse PXRD patterns, so pretty similar to mass spec data.
Glad it was helpful! Interestingly, I have used many of these scripts developed for the videos on X-ray scattering data myself!
I have just found your signal processing series. Excellent. I have a question that I hope you can shed some light one.
I use SciPy fft, fftfreq, and find_peaks, and have found that the amplitudes of the peaks found are hugely different (i.e. fft amplitude of a peak is 1200 and find_peaks has an amplitude of 0.65. Any insight would be appreciated. Thanks.
What techniques do you use to clean your analytical data? 🤓
Can you do one where you find the peaks , global maxima and minima, when the data is non stationary, like in the case of stock prices? Thankyou for this video!
Foward looking is alright, look ahead is fine as my usecase is different, but identifying them is the goal and I find it difficult. I would appreciate your help. :)
Very interesting idea! Let me see if I can find an interesting dataset to try this on.
First of all, I would like to thank you for your explanation..
I have some questions for this method. My experimental data has 1600 information about modal analysis ( frequencies Vs Magnitudes )
So, how can identify the most good peaks with this PPM?
(I'm not good in python)
Hi Somaya! a couple questions to guide the answer. How are you defining “good peaks” tallest peaks? Best resolved? What is the nature of the data? Scipy has a lot of tools for what you’re generally describing but which specific combination of methods will depend on the nature of the data and the questions above.
Happy to help more!
i am working on some protein mass spectra need help how can i connect with you , your video tutorials are very helpful for a undergraduate like me
Let connect in LinkedIn! Look forward to it
Hi , i have the signal FFT of time domain, which is divided in segment, i want to find the peaks od each segment before applying clustering method, but i want to take both the frequency and magnitude into consideration in clusters, what should i do, i can see in you are taking just the magnitude of each peak ..
Howdy! Can you say more about out the raw data? Is it in the frequency domain?
@@CJP3 the raw data was just the energy imbalance (generation-load) in hourly interval, it was in time domain , i created segment before applying fft . i took FFT to convert to frequency domain , now i want to find peaks but considering both Frequency and Time . i am sorry for my English if it is still not clear , i can phrase it differently . thank you for the time
Interesting to see your walkthrough and thanks for sharing. I've used script.signal.find_peaks for locating the position of plotted data in scanned images from means across each axis. Ie a0 = np.mean(img_a, axis=0) & a1 = np.mean(img_a, axis=1). Noise is always a problem but I've had luck subtracting a smoothed version from the original.
Whoa! So you take and image, convert it to an array and then do that processing you just mentioned? 🤯
Man, this video is just awesome, but i am really struggling with how do i transform my RAW data into excel files so i can find the peaks in python. I am a coding beginner so i am still learning the basics, and your vid really help me seeing the bigger picture!
Thanks for the kind words! Is your raw data a .RAW file? If so, from what instrument?
@@CJP3 Yes it is a .RAW data, im using an LC/MS instrument to get the data. I've tried to download the MZconvert and MS-DIAL but none of them worked for me. How do you convert your files in your Mac?
Thanks for the kindness, your the best!
Nice video. What are you using to do your diagrams? an Ipad? also, where can I find the data you used so I can follow along?
Howdy! Yep, I use an Ipad + OBS for screen recording. The data work I collected in Grad School. I think I can upload it Github.
How can I detect p,q,r,s,t peak in a ECG data can you please explain?
There are specific python libraries that can detect waveforms like those in ECGs. One such is Biosppy. I’ve never tried it. Good luck!
Thanks for your help
My pleasure! Thanks for watching!!
Is there any robust, mathematically-backed methodology to define the prominence of the dataset without prior knowledge to it?
Hi, you would need to have a metric for the S/N threshold and sufficient signal processing to clean up the data. For that reason EDA and a visual assessment of the data is a good start. Furthermore, you could ML techniques like DBSCAN to estimate peak threshold values but this would still likely require an iterative process.
@@CJP3 Thank you for your shift response. Visual assessment is out of conversation as we try to automate exactly this. I will try the solutions you proposed. Much appreciated
Do you have any suggestions to detect dips?
Howdy! I’d consider processing the signal using 1/x to invert what is considered a peak.
Wonderful video! How can I find the starting point of a peak? It is very important in acoustics analysis.
Howdy! Thank for watching the video. Does this video get at what you're thinking about: ruclips.net/video/wAmlfQqhCh0/видео.html
If not I'll dig more into this idea :)
Wow, will definitely try it! Thanks!
Is there a way to have python count the number of peaks, I'm working with a data set that may contain a couple hundred peaks to a thousand peaks, I'm working on counting interference fringe lines from a laser aimed at an Arduino photo diode detector and I don't want to have to count up the red dots manually.
Hi Connor! It is possible to build a script to have Python count peaks. If using scipy.find_peaks you can measure the line the peak list. Len(peak_list).
However, I think based on your description you have signal - peaks and interference - peaks the same spectrum.
@CJP3 so the data I'm looking at is just spikes plotted voltage vs time.
Hmm that would be an interesting video! Let me see if I can find an analogous dataset.
For some reason on my plot the red marks at the peaks are not showing up
Howdy, did the line plot show up?
@CJP3 when I comment out the scatter plot to check the p1 array thats generated, theres nothing in the array. For the most part I've copied everything line for line except the renaming of the columns and file path
@@CJP3 I figured it out, i had to omit the values in the line 'p1=indexes(df.v.values....)' to 'p1=indexes(df.v...), where v is my voltage values i want to use as the y value reference in the scatterplot.
This Video really helped me a lot, but for a Python Tutorial your Voice and the Music is awkwardly sexy 😅
Haha Thank you for the compliment! What type of data are you working with?
Can you stop the background music, very distracting
Howdy, I took a poll about whether to keep or drop the background music and the jury was split. So instead I looked at the analytics and retention is higher with background music. To compromise I don’t have it in all the videos. Sorry it’s distracting!
The bgm is like reading in a o club that is stupid
lol I don’t use it as frequently in newer videos