Tutorial 11-Exploratory Data Analysis(EDA) of Titanic dataset
HTML-код
- Опубликовано: 22 авг 2024
- Here is the detailed explanation of Exploratory Data Analysis of the Titanic. Finally we are applying Logistic Regression for the prediction of the survived column.
Github url: github.com/kri...
References from : Jose Portila EDA Materials And Kaggle
⭐ Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I've been using Kite for a few months and I love it! www.kite.com/g...
Stats playlist : • Population vs Sample i...
You can buy my book where I have provided a detailed explanation of how we can use Machine Learning, Deep Learning in Finance using python
Packt url : prod.packtpub....
Amazon url: www.amazon.com...
After so much of struggle with my LMS, I was finally able to understand entire EDA in within 30 minutes. Thank you.🙏👍
Is it the inmovidu one?
What ia LMS
@@ytg6663 learning machine shorts
@@ONE-THING-2RAY Where it is?
@@ytg6663 Learning Management System
Doing a job that of True Guru, Ekalavyas are all around and raring for such knowledge-impartation. Thanks much Krish.
You are awesome sir! Not only are you a great mentor, but also a great motivator. Thanks for all the great work you have been doing. Stay blessed!
I am learning this for data analyst but not sure what more should I learn to get job asap.. if you can help please we can connect on instagram
What a beautiful video for a beginner who is just getting his hands on data science.
Loved the video; in fact, the entire playlist gives an amazing approach to the intricacies of Machine Learning. Thank you, Sir.
You are a gem! Making india learn ML. Thank you for all the stuff you do for us. :)
Me trying to understand data analysis with python couple of days ago now
U actually make it simpler and beginners friendly, more unction to function sir
21:30 Median of the passenger age travelling in each Pclass can be calculated using below code instead of looking at boxplot and guessing the number.
df[df['Pclass']==1]['Age'].median()
df[df['Pclass']==2]['Age'].median()
df[df['Pclass']==3]['Age'].median()
good one brother i was thinking the same y to guess it when we can actually calculate it,....
There is a error comes when I want to use sns.countplot. And the error is "could not interpret input 'survived' "
@@tusharmahuri2439 bro copy the heads from the data set and not just type, the language is case sensitive
it is 'Survived' and not 'survived'
You have actually cleared the EDA concept for me, Thanks a lot !!
why 0 and 1 is taken in cols as the indexing of the column is 2 and 5 then why 0 and 1 is taken can you clear
Both imputing and dropping missing values (NaN) is not a good practice with real world data. The ideal way is to derive a new field indicating missing values. 1 for missing else 0. because, sometimes missing value can be a new information in itself.
just sharing some learning from my job :)
Hi please do you mind sharing how to do that here. Or can I reach you via email?
Yes, it depends upon the dataset and problem you want to solve. In this case, dropping the null value is the best possible option in my opinion.
Sir I'm very impressed to see your such amazing video.. Though I am very weak in programming but now I feel like that i should start my programming journey again cause i have someone like u who can explains anything in very simple way
sir i really liked your video.. but according to road map video, you asked us to watch python 1-24 lectures first..in this eda concept, you have mentioned some new words like get_dummies, and few other new words.. stuck with the last 10 mins explaination.. else everything is really clear and understandable.. thanks for all the efforts...
Get dummy are use in pandas
It is basically one - hot encoding..
Encoding techniques are used to convert categorical data into numerical data
Since it is applied on 'Embarked' column
ruclips.net/video/OTPz5plKb40/видео.html
This is one of the best data set being used to understand how to fix the nulls. Great Job and thank you .
Krish, This material is FIRST CLASS. Appreciate it very much.
You are amazing brother. Your videos are helping me gain confidence in ML. Keep up the good work
Thanks a lot Sir... You've expailed it in a great way... Love from Pakistan
awesome explain ...........I really can't stop myself to comment on this video...……...on of the grt video on data visualization
loved your video , far better than the uni teachers :P
Can u please make a video on treating the outliers, this will help us a lot in solving the problems
Thank you for providing knowledge in a simple way.
Nice work Mr. Krish...... It's really helpful
Very helpful..... U did a lot of hard-work for us.... Thnk u so much sir🙌🙌🙏🙏..... And ur way of teaching is very good that is form basic
I have seen some of your videos, excellent work. I really appreciate your work Mr. Krish Naik.
This video is amazing. Thanks so much for sharing your wealth of knowledge.
this was really one of the most usefull stuff avialable !!!!!!!!!!!!!!!
Great work sir, learning a lot from your videos, please upload more videos on EDA..
Awesome tutorial on Exploratory Data Analysis ❤️❤️
Thanks a lot for the very detailed lesson Sir.. that was really fruitful and helped me complete one of my project. Thanks a ton..
sir how you get to know the age age has relation with pclass (how and which analysis you did?)
@Vinayak sharma you can relate any column with any other column.
You could do a heat map of all features and get their correlation according to which you can know which feature is dependent on what
@Krish You are doing an amazing job.
Great one Krish. Basically covers most of the things a beginner needs to understand.
Is there a part 2 and 3 for this video, about feature engineering on the same dataset?
@Krish : Arrange your Complete ML playlist videos into a roadmap playlist, from start to end : to data scientist
Thanks for the detailed video. Really helpful :)
this is beyond amazing....amazing place to learn and to revise the impn techniques
Very nicely explained. Awesome
I didnt understood why categorical features disappeared in training data for logistic regression
Superb explanations..
And interesting to learning
Krish - Thank you for the EDA,
Throw some light on Story Telling - If you had to conclude the EDA, Theorotically, In lay man terms - we must do the story telling- Correct me If I am wrong .
great video :)
i have a suggestion
we can drop PassengerId to increase the accuracy score because it doesn't contribute to the dependent variable
@naveen rawat
There is a error comes when I want to use sns.countplot. And the error is "could not interpret input 'survived' "
@@tusharmahuri2439show me the line of code
Very nice one thank you very much for sharing valuable information
great job sir, please do make more such videos for practising for beginners .
Great video Sir, I just have two doubts that why did you not use get_dummies on "Pclass" as it was also categorical data.. and second why did you not normalize the "Fare" and "Age" Columns as their values are might over power the results?
Same doubt bro
If you type "train.info( )" you will see thae dtypes of all the columns. I don't know if this might help or not but get_dummies( ) can be used for objects only i think as they do not represent any numerical value for the system to compute get_dummies( ) changes indicates those objects into numerical values. Please correct me if i am wrong as i am also confused about this if you agree or have a different insight on this please tell me so.
Another great video very useful one bro like NLP.. 📍
Hi Krish,
Please create some more videos on EDA, it will be helpful.
Hi, Enable auto subtitle, It helps a lot.
Thank you.
Great work sir!!👍🏻👍🏻
This was so helpful. Thank You
loving the playlist :)))))
Really helpful, Thank you soo much.
Pretty nice explanation !
A very good about EDA but one thing i must mention that you didnt even touch the outliers concept. Its the major part of EDA and honestly i take this video only for outliers . But didnt find .
wonderful explaination
one note, in boxplot the middle line inside the box is median value, not the mean value
i have a question why is he not using SimpleImputer class from scikit learn
instead of finding the realtion to make the nan values having some values
we can easily do it through sklearn module
and also why isnt he using label encoder for binary values ???
kind of fantastic video bro, but it needs 2-3x watch for crystal clear understanding.
I wish I found this the first time around
Great to understand. thanks alot
Thanks a lot Kris. EDA was well explained. I could not understand the last part starting from confusion matrix and how to read the final result of the analysis?
Thankyou sir it is very helpful 😊.
very knowledgeable,thanks man :)
hey @Krish! Should we do this data visualization for each and every column? or we do it after feature selection? if we are supposed t do for each column, wouldn't the code get to big and complex for data with hundreds or thousands of features?
How do you know for one kind of result, which plot to use exactly?
Sir One question always revolves always in my mind that how should we remember all the libraries and syntaxes that are needed to Preprocess the data or doing the visualization stuffs??! It would be grateful if you share your strategies regarding that?!
Same question
@Rahul Ranjan i'll try too thanks for suggestion
when i try to apply my functinon (23:20)it is showing unexpected EOF while parsing
very helpful for beginners
Input contains NaN, infinity or a value too large for dtype('float64'). after logmodel.ft(x_train,Y_train).... any solutions
Actually here male,q and s column contain 884 null values....so here if we remove these columns from train then can remove this error either we can use some statical concept mean mode to remove this...u can try this..hope u will be able to find your answer
Everytime, I import data it shows error "file not found"
import pandas as pd
data=pd.read_csv('C:\Users\Siddhi Singh\Desktop\Iris.csv')
print(data)
Actually you should reset the laptop because if any file found in name of panda means error willl be encountered and in the other case you should download and upload in jupyter notebook and in that jupyter notebook you should copy the path...
wish..... Jack and Rose could also see this data analysis
I think there is no need for "passengerId" to be included in the model. It should be dropped as well.
One correction Sir-- In the boxplot, them middle line is the median(50% percentile). Thank you
20:20 hey, uhmm.. 50% percentile gives us MEDIAN of the age of people with 1st class... So we are using MEDIAN value instead of MEAN right?
Very helpful video for me to understand EDA
You're right, 50%ile is the median. I think you should check out the definition of median and percentiles on this page - www.statisticshowto.com/probability-and-statistics/percentiles-rank-range/#:~:text=The%2050th%20percentile%20is%20generally,quartiles%20is%20the%20interquartile%20range.
That should clear your doubt.
Thank you Krish.
after 28.1 min, krish you just ran quickly ,before it was good .
He was quick becoz later part was related to model building and accuracy which is not the topic of this video..This is all about EDA
Sir, what is the need to visualise the data in this problem. You haven't use any analysis extracted from the visualisation to get help out in data cleaning.
Sir play list is best
But please share the link from which u downloaded dataset fir every vedio
So that we can do what u explained in vegio
I understood till splitting training and output data. But From there Logistical regression application and confusion matrix application is very difficult to understand. I found theoretical explanation of Logistic regression but the code and explanation of syntax and its application videos is not found. Could anyone help with links to understand these two concepts. Thankss
Thank You So Much
3:35 the add🫠💀
Thanks Krish
Good One!
Sir, I didn't get why you compensated the missing value of age with the average age of Pclass?
Can't we simply replace the NaN values with the median values of the age column as: train['Age']=train['Age'].fillna(train['Age'].median())
In practical reality, every person has an age value but that data is missing for some people in the titanic dataset. Our goal is not just to fill in any random age where the age is missing but to fill in an educated guess/ estimate of the missing age of a person so that it can be a close representative of the true ages of those people. Of course, like you mentioned, the median of the entire age column could be used as an estimate but would that be a good representative value for ALL missing ages? Some people would have ages far above or below the median age. So on further exploration we notice that the median age for each Passenger Class is different, which would mean that in reality, people from a certain p-class would more likely be of a certain age, than someone who belongs to another p-class. And this difference is considerable (37 vs 29 vs 24). So by using using p-class to estimate age, we're just making a more educated guess for missing age values. You could of course go several steps further and consider other factors (like maybe SibSp, Parch etc.) in order to get a higher probability age value.
sir im confuse coz we are predicting survival so it is 0 and 1 which means means its a categorical data and we r solving with regression
Hi krish,
You didn't drop the passenger ID column before fit the logistic regression model cause it doesn't contain any information.
I like the video, but how did you know exactly the graphical representation to use, i mean why countplot why not jointplot? Why line plot not boxplot?
I hope you really understand my questions sir
There is a error comes when I want to use sns.countplot. And the error is "could not interpret input 'survived' "
why did you decide to analyse age with respect to Pclass in the missing value stage ?
for me lbfgs is failed to converge on the local minima . How to fix it. i believe more categories need to be labeled like Pclass and standard scaler is required for age and fare .
@krish Naik we drop the age column then how again age column occur
why you place this video in the playlist at this point, there are so many doubts
Thank you sir
@Krish Naik what is that test size =0.30 why did u use that .from beginiinng of video everything was very good but in the end i couldn't understand x train ytrain test size whats that accuracy 0.7190 etc. please tell me sir else your efforts will go waste ...
help plz.......i want to fill nan of zip based on my other feature city. like city X has zip XXX so this nan of zip is having city X then how to fill it with zip XXX??;.
I would suggest to get the average of all the zipXXX known and than copy those same values to the unknown zipXXX of city X
It was a resourceful video.
But why EDA is done before train-test split ?
Input contains NaN, infinity or a value too large for dtype('float64').
i am getting this error while applying logistic regression please help me out
I'm also getting the same problem, did you able to resolve it?
Best explanation
Please upload video related time series analysis
Great video Krish. One doubt- how did you find the correlation between pclass and age before creating the box plot?
Use ANOVA test for finding relationship between variance of each class-group of the categorical variable and the mean of the continuous variables associated with each group.
Thank you very much!!!
The middle line in box plot is not average but it's a median.