Hello, at minute 24:24, I managed to reverse the range of column names using [5:13][::-1]. The expression [::-1] is used to reverse ranges and it is very useful: df2 = df.groupby('Continent')[df.columns[5:13][::-1]].mean(numeric_only=True).sort_values(by='2022 Population', ascending=False) df2 Thank you very much, Mr. Alex, for these tutorials.
This is absolutely top tier content. I can't stress this enough to people new, or going into the DA/DS field: you WILL be exploring and cleaning data sets much more than you will be visualizing and building models. Thanks for this, Alex!
We can also write this to save time writing all the column names in the list "df2 = df.groupby('Continent')[df.columns[12:4:-1]].mean(numeric_only=True).sort_values(by='2022 Population', ascending=False) "
Oceania is one of the 7 Continents (North America, South America, Europe, Asia, Africa, Oceania, Antartica). It's basically Australia and the countries (islands) around it. Hope that helps!
EXCELLENT SUPERB video!! I can't believe it--I'm 6/7 videos away from the end of your FANTASTIC bootcamp series! Wahoo! I learned a lot in this video. :) As for "ending on a low note", hardly Alex lol All your content is uplifting and rewarding! As always, THANK YOU!
Alex, thank you for this great video and everything you do! In order to avoid manual ordering of the population years, there is a way to use df.columns method, by adding reversed. The whole construction looks like df2 = df.groupby('Continent')[list(reversed(df.columns[5:13]))].mean().sort_values(by='2022 Population', ascending=False) And it works )
Hello, Alex! Once again, thanks a lot for all your hard work! At 13:10 I got an error ValueError: 'box_aspect' and 'fig_aspect' must be positive" Solved it by putting the plt.rcParams BEFORE the sns.heatmap The other problem was that some functions didn't work until I added the parameter numeric_only = True, e.g., df.corr (numeric_only=True) or .mean(numeric_only = True) Hope, it can help someone!
at 24:00 you can just simply add ".sort_index()" on the "df3 = df2.transpose()", so that we don't have to manually rearrange the columns. df3 = df2.transpose().sort_index() worked on my end, hope on your end too.
For those get error in heatmap: import matplotlib.pyplot as plt numeric_columns = df.select_dtypes(include=['float']) sns.heatmap(numeric_columns.corr(), annot=True) plt.rcParams['figure.figsize'] = (20, 7) plt.show()
24:50 df2 = df.groupby('Continent').mean(numeric_only=True).iloc[:, -5:-13:-1].sort_values(by = '1970 Population', ascending = False) df2 = df2.transpose() df2.plot() This way we don't use the copypasting and changing columns, just use reversed indexes)
I got some error's (using pycharm) that I solved by using "mumeric_only=True". For instance: df.corr(numeric_only=True) and df.groupby("Continent").mean(numeric_only=True)
Hey use this code instead numeric_df = df.select_dtypes(include='number') # Select only numeric columns plt.figure(figsize=(20, 7)) # Set the figure size sns.heatmap(numeric_df.corr(), annot=True) # Create the heatmap with annotations plt.show()
If you are having any at 16:59, that's because you are looking for the mean of other datatypes apart from numbers. To resolve this, include numeric_only=True in your mean function, as in "df.groupby('Continent').mean(numeric_only=True)"
For the grouping data I do df2=df.drop(columns=['CCA3','Country','Capital']) df3=df2.groupby('Continent').mean(numeric_only=True).sort_values(by="2022 Population",ascending=False) df3 to get to the same output as seen in the video
Hey, just a quick note here, when we're plotting the populations, it's only related to the numeric values compared to the highest populations, in fact (for example) Oceania's population increased in around 2.5 times Anyway, thanks for the content, it's amazing
26:00 you can just add this to inverted columns df2 = df.groupby('Continent')[df.columns[5:13]].mean(numeric_only=True).sort_values('2022 Population', ascending=False) df2_inverted = df2.iloc[:, ::-1] df2_inverted
Thank you so much Alex, truly great content you put out there. I have a question please; when I run df.groupby('Continent').mean() and df.corr() I get errors, please what could be the cause and what can I do to remedy it.
Hi, Alex. First of all thanks for a great video and explanations in it. If you could help out with the issue I get running your exact code I would be more than grateful. Running the df.corr() line gives me the following error: ValueError: could not convert string to float: 'AFG' . Same comes for the heatmap,etc. What could it be here? Thanks a lot in advance.
at 11:12 the df.corr() does not work now. Instead use: df_numeric = df.select_dtypes(include=[float, int]) correlation_matrix = df_numeric.corr() correlation_matrix
another way to select the columns (think of a big data sets where indicing with numbers would be challeging) columns_to_include_2 = df.select_dtypes(include=['number']).filter(like='population').columns
Thank you very much Alex I'm shifting from Ph to Data Analyst with your bootcamp I had an issue with plt.show() AttributeError: module 'matplotlib' has no attribute 'show' i's deprecated and I counldn't find something sameller and also my chart not showing numbers 14:10 Best regards
My heatmap doesn’t contain the data values inside them as in 14:18 instead it just shows a heatmap with column values as in the top most band. I have written the code just as shown above df.corr(numeric_only=True) as well as that ‘annot’ but still no data values. Pls Anyone help
hey, can anyone tell if the correlation command is working in vs code? I'm getting a value error in this part. please share the solution if you have one thanks :)
If "df.corr()" doesn't work for the same data set were using in this Video. And It throughs an error : could not covert string to float: 'AFG'. Like this, Try : df.corr(numeric_only = True)
Hello Alex. I read a few reviews on your recommended course on Udemy. People are saying that it is a bit outdated especially the last section. Do you think I should still go for it and the non updated part doesn't matter? Love your content and thanks for everything you do here.
@@octaverius762 Actually it is relevant. Though different countries do have different models and its entirely up to convention. Australia the continent is usually considered the 3 islands of mainland Australia, Tasmania and Papua New Guinea
I have one problem, which is that the table does not display columns starting from "area (km^2)" when we call "df" to view the table, I mean there is no scrollbar for horizontal data, can anyone help for this, please?
I'm a law graduate without any experience or qualifications in data analysis whatsoever but i want to get into data analysis. Will i be able to get a job in this field? and if yes then what possible skills and certifications will help me to achieve the same? please give me some tips and insights it would be really helpful!
Yes, you can, from skills I would prefer mostly analytical thinking, learn probability and statistics, other high math stuff. From certification mr Alex said that Amazon and Tableau certifications, and others will help, but anyways if it's long-term learning certificate, I think it is ok to have it on CV. But the thing that highlites you it is the projects that you have done mostly for your job and I mean not only portfolio projects but another ones to show your uniqueness.
FYI Australia is not a *small* island. Oceania doesn't "mean" anything, it's the name of a continent containing the countries listed right in front of you since you already filtered the data 😂😂
why use anaconda instead of google collab, just curious looking forward in visual tutorial at python and statistics thanks i really need this type of tutorial i am studying cohort analysis and RFM analysis
corr_matrix = df.select_dtypes(include='number').corr() # Then proceed with creating the heatmap sns.heatmap(corr_matrix, annot=True) plt.rcParams['figure.figsize'] = (20, 7) plt.show() I have used this code for heatmap but the notebook doesn't populate the heatmap with individual correlation values rather colored tiles only. please anyone can help?
df.corr() ❌ df.corr(numeric_only=True) ✅ since this posting numeric_only now defaults to False so if using newer versions of panda here is the correction:
If you want research a trend you need equal range ex. every 10 years on x axis. In your case there is 10 years and then 5 years and 2 years. In my opinion this is wrong approach. Am I right? Btw. Nice content :)
This worked for me where df.corr() did not: # Select numeric columns (excluding any non-numeric columns) numeric_columns = df.select_dtypes(include=['float64', 'int64']) # Calculate the correlation matrix correlation_matrix = numeric_columns.corr() correlation_matrix
Continents are mostly a social convention. The english spekaing countries tend to use 7, while spanish speaking countries have a 6 continent model where it uses Oceania and combines North and south America. Australia is the continent but Oceania is a geopolitical convenience. If it was not included most of the pacific isalnd countries would not be associated with a continent. North and South America are another convenience and Central america is only a region by American standards. As an example of how ridiculous it is as a continent, Hawaii would be included if it was independant.
Sir Alex. I am Roshan Dattaram Dhumal I live in India from Mumbai. I want to start my career in data analysis but I don't know how to start and I want to know what steps you have to take to become Data analytics. I would like to request you to please explain to us and give us some steps. Please sir I will definitely do hard work.
try passing numeric only argument. In recent version, default value of this argument has changed to false so it tries to correlate string values as well. df.corr(numeric_only = True)
Incase you are running into an error at minute 11:12, add numeric_only = True to the corr. i.e df.corr(numeric_only = True).
thanks man !
thank you. really helpful!
Thank you
Thanks! Never seen that before!
thank you!!
Hello,
at minute 24:24, I managed to reverse the range of column names using [5:13][::-1]. The expression [::-1] is used to reverse ranges and it is very useful:
df2 = df.groupby('Continent')[df.columns[5:13][::-1]].mean(numeric_only=True).sort_values(by='2022 Population', ascending=False)
df2
Thank you very much, Mr. Alex, for these tutorials.
Thank You!
Alternatively, start counting columns backwards,
df2 = df.groupby("Continent")[df.columns[-5:-13:-1]].mean().sort_values(by='2022 Population', ascending=False)
df2
or df3.plot().invert_xaxis()
thx
Man, “Oceania” was so funny 😂, tks for the class!
This is absolutely top tier content. I can't stress this enough to people new, or going into the DA/DS field: you WILL be exploring and cleaning data sets much more than you will be visualizing and building models.
Thanks for this, Alex!
Thanks Alex! Right now i'm applying to my first DA Job and you have no idea how useful your videos have been for me!!
Hey? How is it going? Did you succed in applying for the job you want?
We can also write this to save time writing all the column names in the list "df2 = df.groupby('Continent')[df.columns[12:4:-1]].mean(numeric_only=True).sort_values(by='2022 Population', ascending=False)
"
Oceania is one of the 7 Continents (North America, South America, Europe, Asia, Africa, Oceania, Antartica). It's basically Australia and the countries (islands) around it.
Hope that helps!
the correction of df.corr() is:
numeric_columns = df.select_dtypes(include=[np.number])
correlation_matrix = numeric_columns.corr
correlation_matrix()
Thanks it works. Why df.corr() not working on me ?
thanks man.
df.corr(numeric_only = True)
worked for me
@@francescab1413 me too mate! Thanks a lot!
name 'np' not defined?
EXCELLENT SUPERB video!! I can't believe it--I'm 6/7 videos away from the end of your FANTASTIC bootcamp series! Wahoo! I learned a lot in this video. :) As for "ending on a low note", hardly Alex lol All your content is uplifting and rewarding! As always, THANK YOU!
I just finished all the videos in you bootcamp playlist few hours ago and I'm excited to do this again..
Alex, thank you for this great video and everything you do!
In order to avoid manual ordering of the population years, there is a way to use df.columns method, by adding reversed. The whole construction looks like
df2 = df.groupby('Continent')[list(reversed(df.columns[5:13]))].mean().sort_values(by='2022 Population', ascending=False)
And it works )
thank you!
df4=df3.sort_index(ascending=True)
df4 at 26:11 as alex is sorting manually you sort the year directly by this command
Namaste! I found your tutorials "Simple, Easy to follow, and To the point". Thanks.
Hello, Alex!
Once again, thanks a lot for all your hard work!
At 13:10 I got an error ValueError: 'box_aspect' and 'fig_aspect' must be positive"
Solved it by putting the plt.rcParams BEFORE the sns.heatmap
The other problem was that some functions didn't work until I added the parameter numeric_only = True, e.g., df.corr (numeric_only=True) or .mean(numeric_only = True)
Hope, it can help someone!
Thank you, You are the Best!
It certainly helped. Thank you, DuckingDuck.
at 24:00
you can just simply add ".sort_index()" on the "df3 = df2.transpose()", so that we don't have to manually rearrange the columns.
df3 = df2.transpose().sort_index() worked on my end, hope on your end too.
thank you
For those get error in heatmap:
import matplotlib.pyplot as plt
numeric_columns = df.select_dtypes(include=['float'])
sns.heatmap(numeric_columns.corr(), annot=True)
plt.rcParams['figure.figsize'] = (20, 7)
plt.show()
thank you
THANK YOU!!!!!! I almost quit for good.
i had that error in corr : " could not convert string to float: 'AFG'"
do you know how to solve this
thanks a lot toygar
@@nassrmohamed278 df.corr(numeric_only=True)
I always enjoy a video from Alex. Making one of the best videos , while some other channels just can be a real headache
24:50
df2 = df.groupby('Continent').mean(numeric_only=True).iloc[:, -5:-13:-1].sort_values(by = '1970 Population', ascending = False)
df2 = df2.transpose()
df2.plot()
This way we don't use the copypasting and changing columns, just use reversed indexes)
Hello,
100000000 thanks for sharing
For the Corealtion part at 11mn
df.corr(numeric_only=True) # pass numeric only param to not having error
Thank you!
thanks
Where would l have been without this video .
I got some error's (using pycharm) that I solved by using "mumeric_only=True". For instance: df.corr(numeric_only=True) and df.groupby("Continent").mean(numeric_only=True)
Hey use this code instead
numeric_df = df.select_dtypes(include='number') # Select only numeric columns
plt.figure(figsize=(20, 7)) # Set the figure size
sns.heatmap(numeric_df.corr(), annot=True) # Create the heatmap with annotations
plt.show()
helped a ton thanks
Thanks for posting! I had to do SHIFT+TAB on the corr() function to find out how to get only numeric values.
thaaaaaaaaaaaaaaank youuuuuuuuuuuuuuuuu
If you are having any at 16:59, that's because you are looking for the mean of other datatypes apart from numbers. To resolve this, include numeric_only=True in your mean function, as in "df.groupby('Continent').mean(numeric_only=True)"
omg I laughed out loud at the "Oceania" part ;)))) Alex is so funny and brutally honest about things he didn't know ;)))
For the grouping data I do df2=df.drop(columns=['CCA3','Country','Capital'])
df3=df2.groupby('Continent').mean(numeric_only=True).sort_values(by="2022 Population",ascending=False)
df3
to get to the same output as seen in the video
Me too, this should be explained, because Strings can not get easy a mean...to long is most the problem!
THANK YOU!!!!!!
Hey, just a quick note here, when we're plotting the populations, it's only related to the numeric values compared to the highest populations, in fact (for example) Oceania's population increased in around 2.5 times
Anyway, thanks for the content, it's amazing
I really enjoyed this introduction to Pandas! Keep up the good work!
25:13 you can apply negative slicing for reversing the plot
Hi Alex
Thank you so much for your support for freshers in the field of data analytics.
Love from India❤❤
26:00
you can just add this to inverted columns
df2 = df.groupby('Continent')[df.columns[5:13]].mean(numeric_only=True).sort_values('2022 Population', ascending=False)
df2_inverted = df2.iloc[:, ::-1]
df2_inverted
Love from Pakistan Alex, Really Helpful and Enjoyable.
I also like the OOPS sound you make 😂😂
Thank you so much for this. I really enjoyed it and learned a lot of what I had forgotten a few years ago.
Thank you Alex this is very helpful.
Thank you for the Pandas class!
I am sure I am going to use some of these tips. Thank you!😍❤
Thanks a lot for this clear cut explanation. Can you make something similar for NLP projects end to end ?
Thank you so much Alex, truly great content you put out there. I have a question please; when I run df.groupby('Continent').mean() and df.corr() I get errors, please what could be the cause and what can I do to remedy it.
use df.corr(numeric_only = True)
@@sabithsaqlain1367 THANK YOU for this!! This was driving me a little nutty. Really appreciate you sharing this. :)
I could not fix the mean() issue.
df.groupby('Continent').mean(numeric_only=True)
@@chriscurtis95 🙏 Thank You!
Thank you for the useful information!
love your videos alexx ;)
Thanks for all you do. I’m loving the bootcamp. Just finished excel project. However, please can you make a video on story telling?
Hi, Alex. First of all thanks for a great video and explanations in it.
If you could help out with the issue I get running your exact code I would be more than grateful.
Running the df.corr() line gives me the following error: ValueError: could not convert string to float: 'AFG' .
Same comes for the heatmap,etc. What could it be here?
Thanks a lot in advance.
Getting the same errors.
try this ==> df.corr(numeric_only=True)
Best to use df.corr(numeric_only=True) to get around this
you saved my life thanks so much @@dustin3320
df.corr(numeric_only = True)
Hope you also make a Pyspark series 🤓
To exclude rank from being display in the numerice data: columns_to_include = df.select_dtypes(include=['number']).columns.difference(['Rank'])
Neat...
Thanks for sharing this content.
Cheers
Thank you Alex💯🔥
at 11:12 the df.corr() does not work now. Instead use:
df_numeric = df.select_dtypes(include=[float, int])
correlation_matrix = df_numeric.corr()
correlation_matrix
Thank u
This is simple and more straightforward
df.corr(numeric_only = True )
Hi, can someone help. When I plot figures that have been grouped, it doesn't show the figure, just says .
21:09 I just figured it out. Simply add another line after the plot, like:
df2.plot()
plt.show()
another way to select the columns (think of a big data sets where indicing with numbers would be challeging) columns_to_include_2 = df.select_dtypes(include=['number']).filter(like='population').columns
columns_to_include_2 = df.select_dtypes(include=['number']).filter(like='Population').columns.difference(["World Population Percentage"]):P
Thank you so much it was very informative.
mean(numeric_only=True)
This is great! Thank you!
Thank you very much Alex I'm shifting from Ph to Data Analyst with your bootcamp I had an issue with plt.show() AttributeError: module 'matplotlib' has no attribute 'show' i's deprecated and I counldn't find something sameller and also my chart not showing numbers 14:10
Best regards
Hi there, did u find the solution to your problem of not showing numbers? I ran into the same problem too.
@@dishanbhandari hey mate, you found the solution?
My heatmap doesn’t contain the data values inside them as in 14:18 instead it just shows a heatmap with column values as in the top most band. I have written the code just as shown above df.corr(numeric_only=True) as well as that ‘annot’ but still no data values. Pls Anyone help
i am also run into same problem :). I still cant find the solution
upgrade your seaborn package
pip install seaborn --upgrade
restart your kernel and rerun all the boxes
@@jDub997D 1000 thanks bruv... bless you
Lets goo!
hey, can anyone tell if the correlation command is working in vs code?
I'm getting a value error in this part.
please share the solution if you have one
thanks :)
Hey, just use numeric_only = True
Is it ok if I use:
pd.set_option('display.float_format', '{:.2f}'.format) instead of
pd.set_option('display.float_format', lambda x: '%.2f' % x)
or even better you can do lambda x: f"{x:.2f}"
If "df.corr()" doesn't work for the same data set were using in this Video. And It throughs an error : could not covert string to float: 'AFG'. Like this, Try : df.corr(numeric_only = True)
same
numeric_columns = df.select_dtypes(include=[np.number])
correlation_matrix = numeric_columns.corr
correlation_matrix()
Hello Alex. Thanks for the video and content. Is there any video for data per-processing?
its spells 'O-Ce-A-Nia' btw
btw thank for this guidance SIr Alex :)
Hello Alex. I read a few reviews on your recommended course on Udemy. People are saying that it is a bit outdated especially the last section. Do you think I should still go for it and the non updated part doesn't matter? Love your content and thanks for everything you do here.
I haven't taken it in a while - worth listening to more recent comments. Could be outdated?
Thank you so much alex
Alex which continent do you think Australia is in 😮
:D
Australia is also a continent tho😂 sometimes ppl will also refere to NZ ans Aus as the "Australias" but Oceania includes the other surrounding islands
@@chefernandez563 Oceania is a continent, Australia is a country. How people often speak is not relevant
@@octaverius762 Actually it is relevant. Though different countries do have different models and its entirely up to convention. Australia the continent is usually considered the 3 islands of mainland Australia, Tasmania and Papua New Guinea
im getting this error on df.corr() "could not convert string to float: 'AFG'" plz help
superb video sir..
Again, thank you were much!
Sir, in your opinion : Jupyter vs Pycharm? Which is better for Exploratory Data Analysis ?
Can I get the dataset for this
where do i get the csv file from?
why is my program when running corr() is not automatically detecting numbers and runs into an error
my heatmap is broken its not showing all the values even if I wrote the annot = True anyone have a fix? i tried almost everything when I hit shift+tab
oh-shee-ana ! you killed me ...
hi, where is the link for the csv format document?
Alex please make video on how to get international remote data analyst job
Unable to use groupby() in 'Continents' its showing an error: agg function failed [how->mean,dtype->object]
Plese help me with this solution anyone
Hey please tell me how to get a discount for the python with pandas course It is too expensive in Indian currency
I have one problem, which is that the table does not display columns starting from "area (km^2)" when we call "df" to view the table, I mean there is no scrollbar for horizontal data, can anyone help for this, please?
Try another browser. Some browsers doesn't support that feature.
high level , thanks
I'm a law graduate without any experience or qualifications in data analysis whatsoever but i want to get into data analysis. Will i be able to get a job in this field? and if yes then what possible skills and certifications will help me to achieve the same? please give me some tips and insights it would be really helpful!
Yes, you can, from skills I would prefer mostly analytical thinking, learn probability and statistics, other high math stuff.
From certification mr Alex said that Amazon and Tableau certifications, and others will help, but anyways if it's long-term learning certificate, I think it is ok to have it on CV. But the thing that highlites you it is the projects that you have done mostly for your job and I mean not only portfolio projects but another ones to show your uniqueness.
Hey alex why we should use python instead of SQl Because SQl is easy
as R user, the syntax of pandas is just weird in compare to tidyverse (dplyr and tidyr)
Thank you!
Was there here an adult ignorant of what Oceania is or is this some inner joke in the channel?
I can't believe this
Also FYI America is just one continent, in case you doubt it
OceanEeeA
FYI Australia is not a *small* island. Oceania doesn't "mean" anything, it's the name of a continent containing the countries listed right in front of you since you already filtered the data 😂😂
why use anaconda instead of google collab, just curious looking forward in visual tutorial at python and statistics thanks i really need this type of tutorial i am studying cohort analysis and RFM analysis
Oceania is the continent that includes Australian and New Zealand.
corr_matrix = df.select_dtypes(include='number').corr()
# Then proceed with creating the heatmap
sns.heatmap(corr_matrix, annot=True)
plt.rcParams['figure.figsize'] = (20, 7)
plt.show()
I have used this code for heatmap but the notebook doesn't populate the heatmap with individual correlation values rather colored tiles only. please anyone can help?
pip install --upgrade seaborn matplotlib
Update seaborn and matplotlib. It worked for me
df.corr() ❌
df.corr(numeric_only=True) ✅
since this posting numeric_only now defaults to False so if using newer versions of panda here is the correction:
I couldn't get seaborn to import... I tried online solutions about installation but it didn't work
If you want research a trend you need equal range ex. every 10 years on x axis. In your case there is 10 years and then 5 years and 2 years. In my opinion this is wrong approach. Am I right?
Btw. Nice content :)
we can do df3=df3.iloc[::-1] to solve the problem with the date order
So there's another EDA acronym in tech other than Event Driven Architecture.
This worked for me where df.corr() did not:
# Select numeric columns (excluding any non-numeric columns)
numeric_columns = df.select_dtypes(include=['float64', 'int64'])
# Calculate the correlation matrix
correlation_matrix = numeric_columns.corr()
correlation_matrix
i am having problem in downloading the file , can anyone help me out
thank you
Oceania ¿!?? ... Tell me you're american without telling me you're american...
I'm messing around don't take it seriously,
Great video!
Am I the only one who knew Oceania was Australia, New Zealand, Samoa and those places😂😂
thanks sir
great
Continents are mostly a social convention. The english spekaing countries tend to use 7, while spanish speaking countries have a 6 continent model where it uses Oceania and combines North and south America.
Australia is the continent but Oceania is a geopolitical convenience. If it was not included most of the pacific isalnd countries would not be associated with a continent. North and South America are another convenience and Central america is only a region by American standards.
As an example of how ridiculous it is as a continent, Hawaii would be included if it was independant.
Sir Alex.
I am Roshan Dattaram Dhumal
I live in India from Mumbai.
I want to start my career in data analysis but I don't know how to start and I want to know what steps you have to take to become Data analytics.
I would like to request you to please explain to us and give us some steps. Please sir I will definitely do hard work.
try passing numeric only argument. In recent version, default value of this argument has changed to false so it tries to correlate string values as well.
df.corr(numeric_only = True)
no explanation.................pd.set_option('display.float_format',lambda x : '%.2f' % x)
OOPs