wow I was so confused about column transformer and why everyone is using that. I was so confused. People usually include that in the encoding videos without any explanation. You are the first person to explain it separately in your series. I am amazed. Thank you Nitish, I will remember you throughout my journey.
one quick question sir we can do this isntead of these sklearn function this way we can arrange and give orders to our data and its fast too easy to understand instant output df.education[df['education'] == 'School'] = 0 df.education[df['education'] == 'UG'] = 1 df.education[df['education'] == 'PG'] = 2 df.review[df['review'] == 'Poor'] = 0 df.review[df['review'] == 'Average'] = 1 df.review[df['review'] == 'Good'] = 2 df.purchased[df['purchased'] == 'Yes'] = 1 df.purchased[df['purchased'] == 'No'] = 0
"Whenever we are doing a project, instead of train_test_split, we should always do k-fold cross validation." Sir, is my thinking correct ?? If wrong, please rectify me.
Thank you for the great video, sir. I wanted to ask wouldn't it be better to encode the data before splitting it? This way we don't have to transform the train and test sets individually.
oe = OrdinalEncoder(categories=[['Poor','Average','Good'],['School','UG','PG']]) when we have this already defined, so why we need to do oe.fit(X_train) I mean, how will it acutally help when all the calculation was done on oe in first line?
Can somebody explain me what order should be followed while doing any ML Project. Like whether feature scaling should be applied first or encoding categorical data should be done etc. Would be extremely grateful if someone can please clarify. Thanx.
ValueError: Shape mismatch: if categories is an array, it has to be of shape (n_features,). Im getting this error after I run oe.fit(X_train) can any one help me to solve this problem
Why do we even do Label Encoding on target column . I mean that is essentially just 0s and 1s right ? So why we just can’t create dummies ? What’s the logic behind using Label Encoding here ?
Doubt 1. Can't we use ordinal encoding and label encoding before X train and Xtest split???? It would have been an easy task to do the encoding before its split. 2. Cant we use replace function of pandas like replace yes and no to 1 and 0, and replace poor , avg and good to some value like 0, 1 2
I have one doubt during train test split we are fitting on training data while transforming both training and testing suppose this was standardization case then if we fit of train data we get mean and variance of that how can we transform test data using this train data mean and var . I just mean test data should be independent of train data there shouldnt be any type of relationship between them to prevent data leakage . So we must calculate seperate mean and variance for both train and test and fit tranform individually? Pls clarify
zy sub samjh aey gia but need a video after Encoding us py Analysis kesi kry ge aur fine result ko kesi again male female or yes and no mi change kry gy, after 2 or 3 video bad uni video py practical project video bi bny, problem zy ha transform data ho gye ab usi py analysis kesi kry? final output kesi pta chly zy male ha?
Does it matter if the output column is ordinal or nominal before applying label encoding? How to do encoding of categorical feature column with high cardinality? Please help me
when ordinalencoding or onehotencoding done using coloumn transformer the output array columns index are changed i mean encoding done on 5th coloumn after tranformation it is appering to be 1st in array after transformation any solution
Hello Sir, Thank you for the session. Can we extend concept of ordinal encoding on numeric column such as Age? Like in your dataset at 11.45, the values of column Age are: - 98, 16, 53, 69, 77. With more number of records we will have more number of distinct values under the column and at maximum we can have 100 values. Thus, if we classify the numeric values into categories will that not help to make our data analysis and ML model better? For example: We can have a category: Teenager for all Age values from 13 to 19., College students: 20 to 23, Young professionals: 24 to 30, Mid age: 31 to 65 and Senior citizen: 66 to 99. And then finally apply Ordinal encoding on these categories since now we will have order among the classified values. It would be very helpful sir to seek your views on the above. Thank you
Sir what will happen if the output is categorical data but nominal, should I apply Label Encoding there also...?? Actually I want to say that If the output data is categorical, may be that Nominal / Ordinal, in both of case should I apply Label Encoding....?? Thank you for this video🙂
The encoders have additional parameters that you can set to decide what to do incase unseen categories are encountered in the test data. You can check the documentation for more details
Q. If we have a big dataset. let's say Housing_price.csv(from Kaggle), then how would I know which column has ordinal data? is there any API to check? @CampusX #CampusX
I purchased a data science course with around 50k fees but even that they are not teaching this level education. You are such fabulous person.. 👍
50K pani main gaye chapak
bhai 50k ka koi course khardta hai kya koi
wow I was so confused about column transformer and why everyone is using that. I was so confused. People usually include that in the encoding videos without any explanation.
You are the first person to explain it separately in your series. I am amazed. Thank you Nitish, I will remember you throughout my journey.
Never knew about Label encoder and Ordinal encoder, I used to apply label encoder on input features, thanks for this hidden insight Nitish Sir.
Sir kya bolo...just itna hi
U r doing great job...with quality conceptual clearity...
This is a great channel. I'm glad I found it. Thank you so much, Sir, for making this so simple.
Best explanation ever 🙏🏻
Well explained. This really helped me to improve my understanding of ML. Thank you sir.
The way you teach is amazing sir , i was trying for months to learn ML , finally i am glad that i found you .
M first time comment kar rha ,, Kosi channel p because info is really precious ,,, quality bole to Nitish sir
Thanku so much for clearing concepts of encoding technique with example. Very helpful n informative video.
You And Krish Naik Sir are the Brahma And Bishnu Of Data Science.
Right sir
you are great bro
bestest video content
explicit and to-the-point 👍
Kabhi-2 main sochta hu agar aap nahi hote to hamara kya hota. Great explanation
The feeling is mutual sir
I dont know why this channel is not famous
one quick question sir we can do this isntead of these sklearn function this way we can arrange and give orders to our data and its fast too easy to understand instant output
df.education[df['education'] == 'School'] = 0
df.education[df['education'] == 'UG'] = 1
df.education[df['education'] == 'PG'] = 2
df.review[df['review'] == 'Poor'] = 0
df.review[df['review'] == 'Average'] = 1
df.review[df['review'] == 'Good'] = 2
df.purchased[df['purchased'] == 'Yes'] = 1
df.purchased[df['purchased'] == 'No'] = 0
Instead of doing this you can use pandas Map function it'll do same thing
sir hamane classes karake bhi hamko yeh concept nahi sikhaee...thank you sir
Thanku so much for clearing encoding concepts. Very good explaination with example.
Sir, you are doing a great job
"Whenever we are doing a project, instead of train_test_split, we should always do k-fold cross validation." Sir, is my thinking correct ?? If wrong, please rectify me.
Yes, or some other form of cross valuation
Thank you for the great video, sir.
I wanted to ask wouldn't it be better to encode the data before splitting it? This way we don't have to transform the train and test sets individually.
Excellent explanation 👍
You are great efforts 👌 a appreciate you god bless ❤
Best explanation ever
You Are Really Great Sir
Great Content
Thank You😀
Thanks Sir for this Amazing Session
Super explanation sir❤
Thank you sir
maine socha ajj video nhi ayegi but thank you
Too good content sir it is helping me alot
recommend these tutorials to aspiring data scientist
finished watching and coding
sir plz make a video on high cardinality categorical value
Nice explanation
Best
It would be great if there is dataset link in aal the videos.
can we perfom label encoding and oridinal encoding before train test split
oe = OrdinalEncoder(categories=[['Poor','Average','Good'],['School','UG','PG']])
when we have this already defined, so why we need to do oe.fit(X_train) I mean, how will it acutally help when all the calculation was done on oe in first line?
At 16:29 why didn't we convert our transformed numpy array to dataframe?
Can somebody explain me what order should be followed while doing any ML Project. Like whether feature scaling should be applied first or encoding categorical data should be done etc. Would be extremely grateful if someone can please clarify. Thanx.
11:18 *Encoding to Categorical Features*
thank you so much
Please make one vedio on neural architecture search
ValueError: Shape mismatch: if categories is an array, it has to be of shape (n_features,). Im getting this error after I run oe.fit(X_train) can any one help me to solve this problem
Why do we even do Label Encoding on target column . I mean that is essentially just 0s and 1s right ? So why we just can’t create dummies ? What’s the logic behind using Label Encoding here ?
Doubt
1. Can't we use ordinal encoding and label encoding before X train and Xtest split????
It would have been an easy task to do the encoding before its split.
2. Cant we use replace function of pandas like replace yes and no to 1 and 0, and replace poor , avg and good to some value like 0, 1 2
Hello sir, which lecture has the introduction to sk learn library?
What if there are null values in education column and then how should we start the process?
I have one doubt during train test split we are fitting on training data while transforming both training and testing suppose this was standardization case then if we fit of train data we get mean and variance of that how can we transform test data using this train data mean and var . I just mean test data should be independent of train data there shouldnt be any type of relationship between them to prevent data leakage . So we must calculate seperate mean and variance for both train and test and fit tranform individually? Pls clarify
zy sub samjh aey gia but need a video after Encoding us py Analysis kesi kry ge aur fine result ko kesi again male female or yes and no mi change kry gy, after 2 or 3 video bad uni video py practical project video bi bny, problem zy ha transform data ho gye ab usi py analysis kesi kry? final output kesi pta chly zy male ha?
why label encoding can't used to transform input columns ?
In label encoding how would the object know whether to map 0 to 'NO ' and 1 to 'YES'. As we didnt specify.
Does it matter if the output column is ordinal or nominal before applying label encoding? How to do encoding of categorical feature column with high cardinality? Please help me
when ordinalencoding or onehotencoding done using coloumn transformer the output array columns index are changed i mean encoding done on 5th coloumn after tranformation it is appering to be 1st in array after transformation any solution
day2-date:10/1/24
loved it
don't we have to convert the array output in dataframe after transformation is done
Not required
I think sir did in previous videos just be make us visualize the numpy array into the better form.
what to do if the number of features are high (ordinal ones)? I have 40 such features
Hello Sir,
Thank you for the session. Can we extend concept of ordinal encoding on numeric column such as Age?
Like in your dataset at 11.45, the values of column Age are: - 98, 16, 53, 69, 77.
With more number of records we will have more number of distinct values under the column and at maximum we can have 100 values.
Thus, if we classify the numeric values into categories will that not help to make our data analysis and ML model better?
For example: We can have a category: Teenager for all Age values from 13 to 19., College students: 20 to 23, Young professionals: 24 to 30, Mid age: 31 to 65 and Senior citizen: 66 to 99. And then finally apply Ordinal encoding on these categories since now we will have order among the classified values.
It would be very helpful sir to seek your views on the above.
Thank you
Sir if there are yes/no data in data then which encoding should be used?
Please clear my this doubt
done
❤🔥🔥
bol sakte hai sir 3:02
in ordinal encoding an error is raised: Shape mismatch: if categories is an array, it has to be of shape (n_features,)
can any one resolve please
Sir then which encoder we should apply on feature which are not ordinal
Sir what will happen if the output is categorical data but nominal, should I apply Label Encoding there also...?? Actually I want to say that If the output data is categorical, may be that Nominal / Ordinal, in both of case should I apply Label Encoding....??
Thank you for this video🙂
👾👾👾
sir csv file are not download
please help me urgent
how to download dataset from your Github ,it is showing "raw file download" and not downloading
please help anyone
Copy the url and load directly in pandas
I got he concept but all information are in array, do we need to convert them into DF and merge to proceed further ?
Instead you can use a transformer
11 20 to 12 10 - why column transformer
Jitni tareef ki Jae Kam h . ..
kaggle pay nhi milra
How to deal with the situation where there are unseen categories in the test data?
its always good to apply encoding without train test split .
The encoders have additional parameters that you can set to decide what to do incase unseen categories are encountered in the test data. You can check the documentation for more details
@@anjushac9307 will check the doc. Thanks!!
sir data set upload kar o
Engg branch should be the example of ordinal data 🤣
Please help anyone
Q. If we have a big dataset. let's say Housing_price.csv(from Kaggle), then how would I know which column has ordinal data? is there any API to check? @CampusX #CampusX
Best explanation ever 🙏🏻
thank you sir