just a note, at 1:19:21 the format = "mixed" isn't really working for me, and it fills the date_born column with NaT values. So, I tried format = "%d %B %Y" and it works
Hi Keith, watching this video and following along. Just wondering if when we got the fillna code from chat gpt if we should have applied that to our original data frame? Loving the content!
not necessarily it depends you see in case of doing the same kind of cleaning for machine learning dropping an entire col can cause loss of data that might have helped in pattern recognition of the ml algorithm so you can use other methods to handle missing values for that case but i think its better to just handle them seperately rather than just drop an entire coln even tho that is a possible approach for smaller datasets so its case by case basis but as i am analysing this dataset now i see a few colns with excessively large amounts of null values so i think its okay to drop them. Cheers
i did chatgpt for the questions that you framed and it is showing same solution , i could have easily done chatgpt rather than seing this video just download the dataset and put some rows of the dataset in chatgpt and put all the frames question they will be same as in this video for 2 hrs, it took 5 min for chatgpt to do..
Yeah, but what are you going to do when ChatGPT can’t save you? You didn’t “easily” do the task at hand… you made someone/something else do it. Maybe data analyzing isn’t your thing. Perhaps consider being a LLM-expert instead 😊
that's what i used : # Parse out dates from Born and Died df['Born Date'] = df['Born'].str.replace(r'in.*','', regex=True) df['Death Date'] = df['Died'].str.replace(r'in.*','', regex=True)
I didn't want to fillna in this specific dataset given that weights are associated with specific individuals. It didn't seem right to try to automatically populate weights for people based on an average weight or something similar. It's okay to have some nan values in your datasets.
Great! For height/weight parts, it's a bit longer, there be some simple solution measure_pattern = r'(?:(\d+)\s*cm)?(?:\s*/\s*)?(?:(\d+)\s*kg)?' df[['height', 'weight']] = df['Measurements'].str.extract(measure_pattern)
![Felix "Unfair" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/Oswujxm2Ag0/mqdefault.jpg)
Thank you everyone who tuned in today!!
I really thank god that I found your channel thanks for sharing knowledge and keep uploading
29:26 - he starts writing some code
Such a great tutorial Keith. Please keep uploading such high quality videos on Pandas and many more
Thanks Keith! I know it takes some time to prepare and record such staff, but please upload more of Python coding!
will try to keep them coming!
Fabulous session. Thanks Keith 👍
watching from Zambia 🇿🇲
Great stream this was very helpful! Keep up the good work!
My man 💪
we need more like this videos and work on real world data
Can you do an update on the numpy video, thank you so much for these videos it helped me a lot ❤
just a note, at 1:19:21 the format = "mixed" isn't really working for me, and it fills the date_born column with NaT values. So, I tried format = "%d %B %Y" and it works
What's your laptop? Cool videos BTW
Please Upload more videos related to data cleaning
感谢你的辛苦付出
不客气
Hi Keith, watching this video and following along. Just wondering if when we got the fillna code from chat gpt if we should have applied that to our original data frame? Loving the content!
okay i need full course on data science
Hawaiian shirt and Twisted Tea! My man
hawaiian shirt yes, but sorry to disappoint just a standard sparkling water I'm drinking haha
@@KeithGalli 😉
Should i always drop the rows containing null values and then perform the further analysis???
not necessarily it depends you see in case of doing the same kind of cleaning for machine learning dropping an entire col can cause loss of data that might have helped in pattern recognition of the ml algorithm so you can use other methods to handle missing values for that case but i think its better to just handle them seperately rather than just drop an entire coln even tho that is a possible approach for smaller datasets so its case by case basis but as i am analysing this dataset now i see a few colns with excessively large amounts of null values so i think its okay to drop them. Cheers
thank you
Where can we find the dataset?
Amazing thank u sir
Thank you man
you're welcome!
Thanks a lot man
you're very welcome!
HATS OFF TO YOU BRO..........BRING SOME REAL LIFE PROBLEMS AND END TO END PROJECTS RELATED. TO DATA SCIENCE
From Madagascar
why are you drinking soda Keith Galli
It's a sparkling water! No sugar or calories :)
holy fuq
😎😎
Hi Keith,
This code handles the issue will:
# Split column 'Measurements'to height_cms and weight_kgs
dfCpy['height_cm'] = None # add a blank column to store height
dfCpy['weight_kgs'] = None # add a blank column to store weight
# Extract height and weight information
dfCpy['height_cm'] = dfCpy['Measurements'].str.extract(r'(\d+) cm', expand=False).astype(float)
dfCpy['weight_kgs'] = dfCpy['Measurements'].str.extract(r'(\d+) kg', expand=False).astype(float)
dfCpy
i did chatgpt for the questions that you framed and it is showing same solution , i could have easily done chatgpt rather than seing this video just download the dataset and put some rows of the dataset in chatgpt and put all the frames question they will be same as in this video for 2 hrs, it took 5 min for chatgpt to do..
If you're not going to support people efforts, at least don't disappoint them
Yeah, but what are you going to do when ChatGPT can’t save you? You didn’t “easily” do the task at hand… you made someone/something else do it. Maybe data analyzing isn’t your thing. Perhaps consider being a LLM-expert instead 😊
that's what i used :
# Parse out dates from Born and Died
df['Born Date'] = df['Born'].str.replace(r'in.*','', regex=True)
df['Death Date'] = df['Died'].str.replace(r'in.*','', regex=True)
you not use .fillna on your df code?
df['weight_kg'] = df['weight_kg'].fillna(df['height_cm'])
I didn't want to fillna in this specific dataset given that weights are associated with specific individuals. It didn't seem right to try to automatically populate weights for people based on an average weight or something similar. It's okay to have some nan values in your datasets.
Great! For height/weight parts, it's a bit longer, there be some simple solution
measure_pattern = r'(?:(\d+)\s*cm)?(?:\s*/\s*)?(?:(\d+)\s*kg)?'
df[['height', 'weight']] = df['Measurements'].str.extract(measure_pattern)