Why its always like, that i feel you are reading my mind. When you explain something, i get something in mind "what will happen if I do this" and just the very next moment you explain the same. God bless you, teachers like you have a different place in heaven
The clarity you provide is incredible. Please know that you are helping a lot of people by making these concepts easy to grasp. I hope you continue to provide this amazing content! You''re doing a wonderful service for the data science community.
Wow, thank you so much for your incredibly kind comment! I greatly appreciate it. I will do my best to continue creating useful content for the community!
You are very kind! Thank you. I think the answer is this: I have a lot of experience teaching in the classroom, I spend a ton of time preparing for every lesson (or video), and I remember what it's like to not understand something (as well as the mental path I took to gain that understanding). Plus, I care about my students, even those I will never meet! :)
Man, I've been trying to watch a video like this for a long time, with these explanations. I never understood why square brackets, parentheses in these panda searches, now it's totally clear in my mind. Congratulations and thank you.
ive watched over 200 hours of python tutorials and you are by far the best. im going back to watch every single one of your videos because i still learn new things from you.
Just want to say thank you for these videos. I was having so much trouble conceptualizing all this, being new to Python as well. You've helped tremendously. Definitely have a knack for explaining things for a general audience and not going over anyone's head.
the awesomeness of your teaching and will to share your knowledge has been already stated by several comenters, therefore I just want to say thank you!
Great service . It shows that you take a lot of time and effort preparing these videos and uploading them. Excellent work. You have set an example. would like to see your profile.
Excellent as always !! I have already recommended your videos to my colleagues :). Something I am not able to find any where on youtube is how to use If Else to add new columns in a dataframe. Example: Based on Genre, I create a new column: If Genre = 'Drama' then New_Column = 'Col_Drama', Else if Genre isin(["Thriller","Horror"]) then New_Column = 'Col_Thril_horrer".
Great video Kevin! Here is a related question: what is the difference between a view and a copy? Should you ever deliberately use a view instead of a copy? What are the pros. and cons of a view vs. a copy? One motivation for this question lies in understanding the following error when creating & replacing columns/values in a dataframe: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_index,col_indexer] = value instead It looks like you were planning to do a video relating to chained indexing, and so it might be that you were already planning to cover this topic.
Glad you liked the video! Regarding a DataFrame view versus a copy, the difference is simply whether they share the same space in memory (view=same space, copy=different space). It's sometimes not obvious whether you are looking at a view or a copy in pandas, and many times it doesn't even matter. However, you will always want to explicitly make a copy of the DataFrame if you are going to modify it. My upcoming video (August 2) will discuss this more, including how to deal with the warning message you mentioned. In the meantime, here are some relevant links: tomaugspurger.github.io/modern-1.html stackoverflow.com/questions/34884536/what-is-the-point-of-views-in-pandas-if-it-is-undefined-whether-an-indexing-oper
Here's the video I just released, that I think will be helpful in answering your questions about chained indexing: ruclips.net/video/4R4WsDJ-KVc/видео.html Let me know if it helps! :)
I know this might be a bit too late, but a good way to explain the booleans might be with a Vern's Diagram. It is explicit and really easy to represent. Nevertheless, amazing work and this has been helping me a lot more than anything I have from my teacher.
THANK YOU for this video. I've been trying to figure out how to filter multiple columns by a list and could not figure it out. Using the | got it done for me! Thank you thank you thank you
Hey, thanks for the great content. just thought id comment that you can probably simplify this using the DataFrame.query method. Instead of saying movies[(movies.duration >= 200) and (movies.genre == 'Drama')] , you could use movies.query('genre == "Crime" | duration > 200'). That saves you having to use the numerous brackets terms and from having to name the dataframe repeatedly. I think it's also meant to run much more efficiently since it uses the same method as df.eval. Thanks again for the series and particularly for varying it up and not just using the iris dataset like so many other channels do!
You're very welcome for the video series! I enjoyed creating it. Regarding query, I am familiar with that method, but I don't use (or teach it) since the pandas documentation lists it as "experimental": pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-query If that method becomes a permanent part of the pandas API, then I will definitely start using it. Until then, I prefer to use notation that is "stable" (meaning it will definitely remain part of the API).
Just watch this clip to refresh my pandas knowledge. Thanks a lot for these wonderful tutorials 😚. Please when would you be making new videos, I can't wait
Great lecture.. I loved it ! My question is .. For multiple conditions on one column, you have used isin method. What if I have multiple conditions on multiple columns ? How can I write in a concise manner
First of all, THANK YOU SO MUCH for working on this series as well as the series on ML with Python. I have learned from from you than all the other resources combined. I have a question though. Why can't we use 'and' instead of '&' ?
You're very welcome for the videos! Regarding your question, it's a technical detail that was explained to me once before, but I don't fully remember the explanation... sorry!
Python's and, or and not logical operators are designed to work with scalars. So Pandas had to do one better and override the bitwise operators to achieve a vectorized (element-wise) version of this functionality. exp1 and exp2 # logical AND exp1 & exp2 # element-wise logical AND using "and" You are implicitly asking Python to convert to a boolean value. NumPy arrays (of length greater than 1) and Pandas objects such as Series do not have a boolean value -- in other words, they raise ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all(). stackoverflow.com/questions/21415661/logical-operators-for-boolean-indexing-in-pandas Hope this helps...
One question please: what if, in minute 5:23 , the name of the collum was like “A.genre” or “33.duration” ? We would have to put on brackets then? Thank you so much for this videos!!!
Great delivery and style Kevin well done and keep them coming. This session highlights what I dislike about Python. Why would you utilise 'and' within the language itself but '&' in Pandas ?. Why force bracket use when for the last 50 years we have had a clear well understood order of precedence where brackets are used only when needed. I do wish Perl had got its act together when it comes to data mining. No reflection on this series Kevin which is excellent.
Thanks for your kind words about the videos! And, your point is well taken. There are pros and cons to every language, and for many people, the pros of Python outweigh the cons.
As a University Computer Science lecturer for 27 years I have always been interested in what makes a good lecturer. Can I ask, did you model your style on anyone in particular, maybe from your student days or would you say your style is a simply a reflection of you.
Python's and, or and not logical operators are designed to work with scalars. So Pandas had to do one better and override the bitwise operators to achieve a vectorized (element-wise) version of this functionality. exp1 and exp2 # logical AND exp1 & exp2 # element-wise logical AND using "and" You are implicitly asking Python to convert to a boolean value. NumPy arrays (of length greater than 1) and Pandas objects such as Series do not have a boolean value -- in other words, they raise ValueError when used as a boolean value. That's because it's unclear when it should be True or False. ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all(). stackoverflow.com/questions/21415661/logical-operators-for-boolean-indexing-in-pandas I hope this helps...
Hey not sure if you would reply, but I was wondering if these filters can be used on pandastable to show me a filtered dataframe WITHOUT using the in built string query in pandastable gui
Excellent video, very well explained. One question. Isin is for exact match only? using you example if some row has "Crime." instead of "Crime" still consider that row? there is another method to incluid that variation on the write? thank you.
Simply the best, so easy to understand to follow and to learn. Thank you. I have a question, what if I want to know which genres are associated to an actor? I mean, how can I split that column?
wow. that was great you saved me from my struggle. one question though is where do you go to know the command syntax and the explanations for each part? Is there a website os something that you use?
Thanks a lot,great video as usual.I just wanted to ask why exactly can't we use 'and' instead of '&'. I roughly get the idea that and is relational and & is bitwise but still?
Python's and, or and not logical operators are designed to work with scalars. So Pandas had to do one better and override the bitwise operators to achieve a vectorized (element-wise) version of this functionality. exp1 and exp2 # logical AND exp1 & exp2 # element-wise logical AND using "and" You are implicitly asking Python to convert to a boolean value. NumPy arrays (of length greater than 1) and Pandas objects such as Series do not have a boolean value -- in other words, they raise ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all(). stackoverflow.com/questions/21415661/logical-operators-for-boolean-indexing-in-pandas I hope this helps...
@@dataschool thanks a lot... Just love the way you are going... Already a fan! ps: just saw your series of playlists on statistical learning... if possible, do some series on "statistics" and "python libraries on statistics" And also on "deeplearning", "TensorFlow", "GANs" and other advanced topics in the field of Ai & ML Everyone is more of application and using it blindly approach... The theoretical step by step approach in depth is just what I was looking for, and I finally got to you, and am really happy :)
You are great! Wondering if you have a result of a large survey as multiple choice questions how would you best use pandas to clean the data. I would love to see video on that. Thanks!
Hi , i encounter an error as syntax error when using df[ df.col_1 > 0 ] where df is a dataframe with column name col_1 . Any idea , how to resolve. Thanks.
Hi, how to apply multiple filter in loc rows ? ie: i tried movies.loc[[movies.dutation>=200],[movies.genre=='Drama'],['duration',['genre']] but it dint work , i can able to apply only one filter in rows like movies.loc[movies.duration>=200, [ 'duration','genre']]
Try this: filter_criteria = (movies['duration'] >= 200) & (movies['genre'] == 'Drama') movies_filtered = movies[ filter_criteria ] Here is a video explanation that I created on this topic (see 2 min 30 sec specifically) ruclips.net/video/ni9ng4Jy3Z8/видео.html
Dear Kevin Markham, thank you for your videos. They have been most useful. In respect to this video entitled "How do I apply multiple filter criteria to a pandas DataFrame?" how could someone apply a double multy-criteria filtering process? I have been trying to use the the same example you show in this video (using a video dataframe). Here is an example of what I have been trying to do: "filter all movies under Crime, Drama and Action Criteria, whose duration would be larger or equal to 150 min". Here is the line code I tried in pandas, but unsuccessfully. line code: movies[movies.genre.isin(['Crime', 'Drama', 'Action']) & movies.duration>=150] Sincerely,
Quick question - can you apply the isin function together with another function? i.e. could you say: movies[(movies.genre.isin(['Crime', 'Drama', 'Action'])) & (movies.duration >= 180)] ? Thanks for the great videos!
hi if i have two data frames with diff indexes,, same length, and i want to filter data from first using condition on a column of the second. Is it possible to implement in some way?? when i tried..df1[df2[4]>40]...i received this error.......(Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match),,i understand this is because of diff index...but is there a way??
Hi , How to select particular column using mutiple filter criteria like movies[(movies["duration"]>=200) & (movies["genre"].isin(["Drama","Action"]))].title here i can get only title where these conditions are met but if i want to get title and star_rating then how to do that?
It looks like you are on the right track. Just add two square pairs of square brackets [[ ]] with the names of the columns that you want returned. Be sure to put quotation marks around the column names. movies[(movies["duration"]>=200) & (movies["genre"].isin(["Drama","Action"]))][['title','star_rating']]
Why its always like, that i feel you are reading my mind. When you explain something, i get something in mind "what will happen if I do this" and just the very next moment you explain the same. God bless you, teachers like you have a different place in heaven
Wow, you are so very kind Suman! Thank you! 🙏
The clarity you provide is incredible. Please know that you are helping a lot of people by making these concepts easy to grasp. I hope you continue to provide this amazing content! You''re doing a wonderful service for the data science community.
Wow, thank you so much for your incredibly kind comment! I greatly appreciate it. I will do my best to continue creating useful content for the community!
Been coding for years and this is some of the best teaching I've ever seen. Thanks!
Thank you! I really appreciate it :)
Question: How can you be such a great teacher?
You are very kind! Thank you.
I think the answer is this: I have a lot of experience teaching in the classroom, I spend a ton of time preparing for every lesson (or video), and I remember what it's like to not understand something (as well as the mental path I took to gain that understanding). Plus, I care about my students, even those I will never meet! :)
@@dataschool Thank you for caring about us ( your students that u never meet)
Yep
@@dataschool "Plus, I care about my students, even those I will never meet! :)" a few teachers bother to say this ..Happy Belated teachers Day !
@@dataschool we probably may never meet, this line touched me.
I care about my students, even those I will never meet! :)
You are a lifesaver!!! That last tip at the end of the video is exactly what I was looking for. Thank you so much!
You are very welcome! 🙌
I am learning python in a data analytics course, you are awesome. I subscribed straight away.
Thank you so much!
Man, I've been trying to watch a video like this for a long time, with these explanations. I never understood why square brackets, parentheses in these panda searches, now it's totally clear in my mind. Congratulations and thank you.
That's awesome to hear! Thank you so much for your kind words!
My guy you are a lifesaver. Legitimately pulled me out of an onset panic attack involving deadlines at work. Appreciate you so much.
So glad I can be helpful to you! I appreciate your gratitude! Thanks for sharing and sorry to hear about your panic attack!
The best teacher i've ever seen :D Thank you so much!
You're very welcome! :)
Awesome content
@@dataschool thanks in 2019
You’re indeed a very good teacher. I am so impressed with your skill of teaching.
God bless you
Thank you so much!
ive watched over 200 hours of python tutorials and you are by far the best. im going back to watch every single one of your videos because i still learn new things from you.
That's awesome! Thank you so much for your kind words!
Your are the only Python Instructor i have been able to follow until now. Thank you.
You're welcome!
Just want to say thank you for these videos. I was having so much trouble conceptualizing all this, being new to Python as well. You've helped tremendously. Definitely have a knack for explaining things for a general audience and not going over anyone's head.
That's awesome! Thanks very much for your kind words!
the awesomeness of your teaching and will to share your knowledge has been already stated by several comenters, therefore I just want to say thank you!
Thank you so much! 😊
I have to say that you're an EXCELLENT teacher, greeting from Turkey! :)
Thank you so much!
This short video has cleared up 2 semesters of Grad School. Thank you!
That's awesome to hear! 🙌
Best teacher i have ever seen .its a pleasure to watch your videos
This man has a solutions to every weird question that comes in my mind.
Great!
i am new to Python and Dataframe, you have explained it in a wonderful way to visualize the over all picture of data and way to extract ! thank you
You're welcome!
Thanks for posting this, very easy to follow and incredibly useful to someone just starting out with Pandas. Great work!
Glad it was helpful!
your each video is a masterpiece of learning pandas. Many thanks for your efforts and nobility to share your knowledge...
Thank you so much! 🙏
The final tip in the video would be very handy. Excellent video. Thanks :)
Awesome, glad it was helpful to you :)
I need another video, as clear as this one, illustrating the use of matplotlib.Thanks a lot
Thanks for your suggestion!
I am very impressed of how you exxplain the functions in pandas. It works very fine for me. And +350 others I guess.
Exactly what I was looking for. Explained very well. Thank you!
You're very welcome!
You're a fantastic teacher. Well done & thank you!
You are amazing teacher i didn't find pandas so easy anywhere
Thank you!
Fantastic... you are among the top 0.000001% of the best teachers on earth. Thanks:)
Thanks very much for your kind words!
Great service . It shows that you take a lot of time and effort preparing these videos and uploading them. Excellent work. You have set an example. would like to see your profile.
Thanks very much for your kind words! Here's more information about me: www.dataschool.io/about/
You remind me of Dr Sheldon Cooper! :) Great Effort! :) Keep up the good work!
Ha! Thanks! :)
OMG YESS! I was thinking about whom he reminds me of :D That's right. Sheldon!
Except the fact that Dr. Sheldon Cooper is not at al la brilliant teacher as this guy! :D
Yes! I was thinking about this since the first video :D
I really appreciate your time and effort in these amazing videos. I learned too much
Thank you so much for your kind words! 🙏
You're right, is not easy to explain that way, the best teacher I've ever seen.
Thank you! 😄
Exactly what I needed to solve a problem. Thanks a bunch.
You're welcome!
Incredible videos, the best explanations of pandas I've ever encountered
Thanks so much!
Better than the class i'm in!
Ha! Thank you :)
Thank you so much! You should have 10 million subs! Cheers!
😊
Fantastic video and easy to follow and understand!
Thanks very much! Glad it was helpful to you :)
I really do not understand how people can dislike this video.
Thanks for your kind words! :)
Excellent as always !! I have already recommended your videos to my colleagues :).
Something I am not able to find any where on youtube is how to use If Else to add new columns in a dataframe. Example: Based on Genre, I create a new column: If Genre = 'Drama' then New_Column = 'Col_Drama', Else if Genre isin(["Thriller","Horror"]) then New_Column = 'Col_Thril_horrer".
Thanks for sharing my videos! As for your question, that will be multiple lines of code though I won't have the time to detail it all out. Good luck!
Awesome! Every visit leaves me with value added. Thank you Coach
That's awesome to hear! 🙌
You're Amazing 🙌🙌 !! Sorry for those who is disliked his Video.
If they don't understood from here, nobody can teach them...
Thanks for your kind words!
Best Best Best Teaching! Thanks a lott, Kevin! You are amazing.
Love from India!
Thank you so much!
Great video Kevin!
Here is a related question: what is the difference between a view and a copy? Should you ever deliberately use a view instead of a copy? What are the pros. and cons of a view vs. a copy?
One motivation for this question lies in understanding the following error when creating & replacing columns/values in a dataframe:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
It looks like you were planning to do a video relating to chained indexing, and so it might be that you were already planning to cover this topic.
Glad you liked the video! Regarding a DataFrame view versus a copy, the difference is simply whether they share the same space in memory (view=same space, copy=different space). It's sometimes not obvious whether you are looking at a view or a copy in pandas, and many times it doesn't even matter. However, you will always want to explicitly make a copy of the DataFrame if you are going to modify it. My upcoming video (August 2) will discuss this more, including how to deal with the warning message you mentioned. In the meantime, here are some relevant links:
tomaugspurger.github.io/modern-1.html
stackoverflow.com/questions/34884536/what-is-the-point-of-views-in-pandas-if-it-is-undefined-whether-an-indexing-oper
Here's the video I just released, that I think will be helpful in answering your questions about chained indexing: ruclips.net/video/4R4WsDJ-KVc/видео.html
Let me know if it helps! :)
I know this might be a bit too late, but a good way to explain the booleans might be with a Vern's Diagram. It is explicit and really easy to represent.
Nevertheless, amazing work and this has been helping me a lot more than anything I have from my teacher.
Great ideas, and thanks for your kind words! 😄
THANK YOU for this video. I've been trying to figure out how to filter multiple columns by a list and could not figure it out. Using the | got it done for me! Thank you thank you thank you
You're so very welcome! Congrats for figuring it out!
Hey, thanks for the great content. just thought id comment that you can probably simplify this using the DataFrame.query method. Instead of saying movies[(movies.duration >= 200) and (movies.genre == 'Drama')] , you could use movies.query('genre == "Crime" | duration > 200').
That saves you having to use the numerous brackets terms and from having to name the dataframe repeatedly. I think it's also meant to run much more efficiently since it uses the same method as df.eval. Thanks again for the series and particularly for varying it up and not just using the iris dataset like so many other channels do!
You're very welcome for the video series! I enjoyed creating it.
Regarding query, I am familiar with that method, but I don't use (or teach it) since the pandas documentation lists it as "experimental": pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-query
If that method becomes a permanent part of the pandas API, then I will definitely start using it. Until then, I prefer to use notation that is "stable" (meaning it will definitely remain part of the API).
Just watch this clip to refresh my pandas knowledge.
Thanks a lot for these wonderful tutorials 😚.
Please when would you be making new videos, I can't wait
New pandas video coming out next week! :)
Excellent video! Very clear Mr. Pandas :D
Thanks! I like that nickname :)
why did you use bitwise and (&) instead of logical and(and)?
Very clear explanations. Thank you!
You are welcome!
I very much congratulate you for sharing code used in video with us. Many thanks for that. It is very much useful to me. My warm regards to you.
Thanks!
Hi - I don't normally comment on videos but these are fantastic! I thought I'd show some support :) Keep on pushing on!
-Harry
Awesome, thanks for your support! :)
Excellent teacher! Keep training us please!!!
Thanks!
Simply awesome explanation...👌👌
Thank you!
Kevin, you are the best bro
Thank you! 😄
I get alot from your videos. Stay blessed
Thank you!
You are the best teacher! Wonder if you can make a vedio to show how to create decile by an attribute in Python. Thanks
Thanks for your kind words! Could you clarify what exactly you are looking for?
Thank you for helping me. You just saved my project !!
Glad I could help!
what is the simpliest way for and operator? ... just like you tell about and operator at the end...great approach sir.
Great lecture.. I loved it ! My question is .. For multiple conditions on one column, you have used isin method. What if I have multiple conditions on multiple columns ? How can I write in a concise manner
.isin()... Thank you for this tips. I have been searching for this for a long time on how to simplify my search query script.
Great to hear! :)
First of all, THANK YOU SO MUCH for working on this series as well as the series on ML with Python. I have learned from from you than all the other resources combined.
I have a question though. Why can't we use 'and' instead of '&' ?
You're very welcome for the videos! Regarding your question, it's a technical detail that was explained to me once before, but I don't fully remember the explanation... sorry!
Python's and, or and not logical operators are designed to work with scalars. So Pandas had to do one better and override the bitwise operators to achieve a vectorized (element-wise) version of this functionality.
exp1 and exp2 # logical AND
exp1 & exp2 # element-wise logical AND
using "and" You are implicitly asking Python to convert to a boolean value.
NumPy arrays (of length greater than 1) and Pandas objects such as Series do not have a boolean value -- in other words, they raise
ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all().
stackoverflow.com/questions/21415661/logical-operators-for-boolean-indexing-in-pandas
Hope this helps...
One question please: what if, in minute 5:23 , the name of the collum was like “A.genre” or “33.duration” ? We would have to put on brackets then?
Thank you so much for this videos!!!
That's correct. Glad the videos are helpful to you!
Great delivery and style Kevin well done and keep them coming. This session highlights what I dislike about Python. Why would you utilise 'and' within the language itself but '&' in Pandas ?. Why force bracket use when for the last 50 years we have had a clear well understood order of precedence where brackets are used only when needed. I do wish Perl had got its act together when it comes to data mining. No reflection on this series Kevin which is excellent.
Thanks for your kind words about the videos! And, your point is well taken. There are pros and cons to every language, and for many people, the pros of Python outweigh the cons.
As a University Computer Science lecturer for 27 years I have always been interested in what makes a good lecturer. Can I ask, did you model your style on anyone in particular, maybe from your student days or would you say your style is a simply a reflection of you.
It's simply a reflection of me!
Yes teachers are born, rarely made. Well done excellent series
Python's and, or and not logical operators are designed to work with scalars. So Pandas had to do one better and override the bitwise operators to achieve a vectorized (element-wise) version of this functionality.
exp1 and exp2 # logical AND
exp1 & exp2 # element-wise logical AND
using "and" You are implicitly asking Python to convert to a boolean value.
NumPy arrays (of length greater than 1) and Pandas objects such as Series do not have a boolean value -- in other words, they raise
ValueError when used as a boolean value. That's because it's unclear when it should be True or False.
ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all().
stackoverflow.com/questions/21415661/logical-operators-for-boolean-indexing-in-pandas
I hope this helps...
got it ok.your talk is very clear and educative.Thank you.
Thanks!
An incredible effort ...keep it up.
Thanks very much for your kind words!
¡Gracias!
Wow, thank you! That is so very kind of you, Daniel! 🙏
Hey not sure if you would reply, but I was wondering if these filters can be used on pandastable to show me a filtered dataframe WITHOUT using the in built string query in pandastable gui
Hi, when I did the step at 5:21 using my own dataset, it only returns the column names. Do you know why it does that?
Maybe you have a typo somewhere in there? Or you have modified the DataFrame before this step?
Excellent video, very well explained. One question. Isin is for exact match only? using you example if some row has "Crime." instead of "Crime" still consider that row? there is another method to incluid that variation on the write? thank you.
This is great! Thank you so much, I was stuck in this problem before.
Great to hear!
Simply the best, so easy to understand to follow and to learn. Thank you.
I have a question, what if I want to know which genres are associated to an actor?
I mean, how can I split that column?
Thanks for your valuable lesson. May I know which screen video recorder software do you use because clarity is great. Thanks
wow. that was great you saved me from my struggle.
one question though is where do you go to know the command syntax and the explanations for each part?
Is there a website os something that you use?
how do I display the unique values in the dataframe column and inplace them?
awesome video series! Thanks for sharing the knowledge!
You're very welcome!
thanks for the video. What about if you need to check two columns and if the value of the list isin either one?
HiI!
Could you please specify that how can we know that when to use " ( ) " or " [ ] " while running any function or method?
() this is a tuple and [ ] is a list
You are a good teacher
Thank you!
Thanks a lot,great video as usual.I just wanted to ask why exactly can't we use 'and' instead of '&'. I roughly get the idea that and is relational and & is bitwise but still?
Python's and, or and not logical operators are designed to work with scalars. So Pandas had to do one better and override the bitwise operators to achieve a vectorized (element-wise) version of this functionality.
exp1 and exp2 # logical AND
exp1 & exp2 # element-wise logical AND
using "and" You are implicitly asking Python to convert to a boolean value.
NumPy arrays (of length greater than 1) and Pandas objects such as Series do not have a boolean value -- in other words, they raise
ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all().
stackoverflow.com/questions/21415661/logical-operators-for-boolean-indexing-in-pandas
I hope this helps...
Excellent answer, Atanu!
@@dataschool thanks a lot... Just love the way you are going... Already a fan!
ps: just saw your series of playlists on statistical learning... if possible, do some series on "statistics" and "python libraries on statistics"
And also on "deeplearning", "TensorFlow", "GANs" and other advanced topics in the field of Ai & ML
Everyone is more of application and using it blindly approach...
The theoretical step by step approach in depth is just what I was looking for, and I finally got to you, and am really happy :)
@@mratanusarkar Thank you.
Hello sir, is this enough to learn as a data analyst.....
Thank you Kevin. df.Series.isin() method is really impressive!! Please keep up the good work.
Thanks!
Very interesting and well explained, thank you!
You are great!
Wondering if you have a result of a large survey as multiple choice questions how would you best use pandas to clean the data. I would love to see video on that. Thanks!
Thanks for your suggestion!
Your the Awesome Teacher... Its really helpful... Thanks SO Much ..
You're welcome!
Thanks so much! Great video - you are an excellent teacher! :)
Thank you so much!
Dear Sir, in the bonus tip - Multiple Filter , may we use the 'AND' in the 'isin' [ list ] instead of' OR' for the same result.
Thanks.
Not sure I understand your question, sorry!
Hi ,
i encounter an error as syntax error when using df[ df.col_1 > 0 ] where df is a dataframe with column name col_1 . Any idea , how to resolve. Thanks.
Maybe col_1 is not numeric?
I use and in python code. Are we not able to use it here because we are dealing with boolean?
I want to filter 2 or more categories data in a single column so would I do that?
Hi, how to apply multiple filter in loc rows ? ie: i tried movies.loc[[movies.dutation>=200],[movies.genre=='Drama'],['duration',['genre']] but it dint work , i can able to apply only one filter in rows like movies.loc[movies.duration>=200, [ 'duration','genre']]
Try this:
filter_criteria = (movies['duration'] >= 200) & (movies['genre'] == 'Drama')
movies_filtered = movies[ filter_criteria ]
Here is a video explanation that I created on this topic (see 2 min 30 sec specifically) ruclips.net/video/ni9ng4Jy3Z8/видео.html
excellent tips, looking forward to more video from you
Thanks!
Great Job Kevin !
You're very welcome, Scott! Thanks for your support :)
How articulate can one be! Thank you so much.
Thanks very much for your kind words!
Dear Kevin Markham,
thank you for your videos. They have been most useful.
In respect to this video entitled "How do I apply multiple filter criteria to a pandas DataFrame?"
how could someone apply a double multy-criteria filtering process?
I have been trying to use the the same example you show in this video (using a video dataframe).
Here is an example of what I have been trying to do: "filter all movies under Crime, Drama and Action Criteria, whose duration would be larger or equal to 150 min".
Here is the line code I tried in pandas, but unsuccessfully.
line code:
movies[movies.genre.isin(['Crime', 'Drama', 'Action']) & movies.duration>=150]
Sincerely,
I think this should work:
movies[(movies.genre.isin(['Crime', 'Drama', 'Action'])) & (movies.duration>=150)]
Quick question - can you apply the isin function together with another function? i.e. could you say:
movies[(movies.genre.isin(['Crime', 'Drama', 'Action'])) & (movies.duration >= 180)] ?
Thanks for the great videos!
What you are suggesting should work... glad you like the videos!
Data School thanks it did. Have you got any lesson on using the pivot function? Might be one to consider?
I don't yet, but it's under consideration for the future :)
Awesome!!! Very well done!!!
Thanks!
What if you were to filter the title that has for example "Fight" on the title. How to do it?
hi if i have two data frames with diff indexes,, same length, and i want to filter data from first using condition on a column of the second. Is it possible to implement in some way?? when i tried..df1[df2[4]>40]...i received this error.......(Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match),,i understand this is because of diff index...but is there a way??
I think you would have to reset one or both of the indexes so that they match.
Sir you are a great teacher, could you please make video on most important used 'import' based on type of tasks we generally do? thanks a lot!!!
Thanks for your suggestion!
Hi ,
How to select particular column using mutiple filter criteria
like movies[(movies["duration"]>=200) & (movies["genre"].isin(["Drama","Action"]))].title
here i can get only title where these conditions are met but if i want to get title and star_rating then how to do that?
It looks like you are on the right track. Just add two square pairs of square brackets [[ ]] with the names of the columns that you want returned. Be sure to put quotation marks around the column names.
movies[(movies["duration"]>=200) & (movies["genre"].isin(["Drama","Action"]))][['title','star_rating']]