How to Detect and Remove Outliers in the Data | Python
HTML-код
- Опубликовано: 7 авг 2024
- ⭐️ Content Description ⭐️
In this video, I have explained on how to detect and remove outliers in the dataset using python. Removing outliers will be very helpful for data cleaning and preprocessing. The methods used are z-score, inter quartile range & percentile.
Text-based Tutorial: www.hackersrealm.net/post/det...
GitHub Code Repo: bit.ly/datascienceconcepts
🌐 Website: www.hackersrealm.net
🔔 Subscribe: bit.ly/hackersrealm
🗓️ 1:1 Consultation with Me: calendly.com/hackersrealm/con...
📷 Instagram: / aswintechguy
🔣 Linkedin: / aswintechguy
🎯 GitHub: github.com/aswintechguy
🎬 Share: • How to Detect and Remo...
⚡️ Data Structures & Algorithms tutorial playlist: bit.ly/dsatutorial
😎 Hackerrank problem solving solutions playlist: bit.ly/hackerrankplaylist
🤖 ML projects tutorial playlist: bit.ly/mlprojectsplaylist
🐍 Python tutorial playlist: bit.ly/python3playlist
💻 Machine learning concepts playlist: bit.ly/mlconcepts
✍🏼 NLP concepts playlist: bit.ly/nlpconcepts
🕸️ Web scraping tutorial playlist: bit.ly/webscrapingplaylist
Make a small donation to support the channel 🙏🙏🙏:-
🆙 UPI ID: hackersrealm@apl
💲 PayPal: paypal.me/hackersrealm
🕒 Timeline
00:00 Introduction to Detection of Outliers
02:01 Z-score Method
12:00 Inter Quartile Range Method
17:12 Percentile Method
#detectoutliers #mlconcepts #hackersrealm #removeoutliers #anomalydetection #deeplearning #machinelearning #datascience #model #project #artificialintelligence #beginner #analysis #python #tutorial #aswin #ai #dataanalytics #data #bigdata #programming #datascientist #technology #coding #datavisualization #computerscience #pythonprogramming #analytics #tech #dataanalysis #programmer #statistics #developer #ml #coder #dataanalyst
Amazing tutorial. Bro, you made my day. Lots of love from Pakistan.
Glad to hear that!!!
You are from Pakistan !! Amazing !😀
You are amazing bro. Don't have words to thank you. you have cleared my many concepts. Lots of love from UK and god bless you. 😊
Thank you so much for your kind words ❤️
Thank you very much sir!!
This is very helpful. Excellent.
Glad you liked it!!!
This video helped me a lot. Thanks!
Glad it was helpful!!!
what an amazing video
really a great explanation
Glad you liked it!!!
Too good....and simple thanks a lot☺️🙏🏼
Glad you like it sushmita!!!
@@HackersRealm could you suggest some paid internship which I can start off with the very next month
@@sushmitarawat6438 For ML based internship, it's better to compete in hackathons or contest to get internship.. You could checkout hackerearth, techgig, etc., for that
@@HackersRealm ok
Thank you so much!!!. A question: in what type of distributions can the box plot be used? For example, if the data follows a uniform distribution, does it make sense to find outliers? What do you recommend me?
You can use box plot and check if there are any outlier for any distribution. If there is some outliers, do the processing, if not ignore it.
@@HackersRealm thanks for your answer
Very Great.
Glad you liked it!!!
Greate Tutorial!! Thanks a lot!! I have a question that How could I do it with the whole dataset? not a single one
you can iterate the columns and process the whole data
@@HackersRealm So to iterate it we will be using for loop passing each column name as I??
@@aniketlode4808 yeah
Good vedio... Do i need check for all the numeric columns one by one and perform capping operation??????
You can use a loop to do it for all numeric columns at once...
Thank you so much,
I have a question, do we need to do this process for each column one by one?
yes, that's correct, you can use loops to automate this.
What about dealing with categorical columns in the context of outliers?
I don't think there will be outliers in categories
what do you think is the best method out of these three ?
You can use any method as it's producing similar results, but instead of deleting samples, trim it in the range
Yo bro I m also learning ai and ml concepts I just need to work one some project or get the training in this
Plz tell me if you can help
check the iris dataset analysis project in the playlist for start
Bruh I have a doubt..... please explain briefly..... These three techniques are used for trimming or capping outliers in the dataset...... But why don't we use only z-score to find outliers. Then what's the diff between these three techniques??
Hii..my dataset has 19 columns and at least 10 colums shows outliers..
So do I have to perform this process for every column each time?
Yes it's better to do the process in a loop and fix it for better results
@@HackersRealm Can you kindly show this process too. Searching for it everywhere can't find it.
@@avashchand9623 what process you're referring?
@@HackersRealm I think he is asking for the process of looping the columns
Pls after I have handled each column outlets how do I save it and which data frame should I continue using
Which method is the most preferred?
It's not about preference, it depends on where and which use case you're trying to solve
@@HackersRealm if there are 40% outlier then ?
@@madhulikasuman2803 it depends on the nature of data, need to understand the domain, and see why this is the case. We could do some data transformation like log transformation to change it
why you decided to use residual sugar as a column to find outliers? any tips and tricks on which columns should be used to find outliers within the dataset?
we can use boxplot or violinplot to find the outliers. You can see some dots outside the line which can be considered as outliers.
8:35 outliers=26
My df is empty while finding the outliers. Any idea why it is so?
which cell you faced the issue?