Starting in pandas version 0.18.1, you can create a new datetime column directly from a DataFrame, based solely on the column names! It's a useful trick, which I explain in this video: ruclips.net/video/-NbY7E9hKxk/видео.html
I was actually going to ask you about this. I started learning Pandas thanks to your video series and I feel like I learnt a lot. Thanks for all the awesome videos.
Actually, I still need some help. I want to create a column for "year/month" from datetime column. For example, if datetime is 2016-07-06 15:56:19, I want to map "2016-07" into a column. Of course I can get first seven characters by converting it to a string, but what is the correct way to do this? I have following the following lambda function: get_month = lambda x: '{}-{:02}'.format(x.to_datetime().year, x.to_datetime().month) my_df["year_month"] = my_df["timestamp"].map(get_month) For now, I do it like this but I am sure a better and more efficient way exists for the job. I'd be glad if you can help!
If your desired end result is a string (such as '2016-07'), then I think using string methods is the way to go! Perhaps something like this: ufo.Time.dt.year.astype(str).str.cat(ufo.Time.dt.month.astype(str), sep='-') However, there is probably an even simpler approach that I'm not thinking of...
Hi, I know its a year late, but in case you (or anyone else is interested) "pd.Series.dt.strftime" is an easy way to output dates as strings in whatever format you like. pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.dt.strftime.html#pandas.Series.dt.strftime
I've been watching your videos for a while now but never got the chance to comment on it i just want to say keep up the great work! You are just awesome!
this is an awesome video but the to_datetime is not working for me, it keeps giving me error like "hour must be in 0..23: 10/11/2006 24:00" I've tried everything
I have a column containing times in the format hour, minute, second (e.g. 00:24:43) and are currently an 'object' type. How can I calculate the average time for this column? I have tried converting to a pandas datetime but this throws up several different errors when I try to calculate the mean? Also, I have three separate groups (full match, T1 and T2) how can I use groupby to figure out the mean times for each group? Thanks
Thanks for your great video! It seems pandas.Series.dt.weekday_name is removed in pandas version 0.23.0. and one can use pandas.Series.dt.day_name() instead.
@Data School , Mark, you are just amazing. :) You make it appear everything quite simple. My humble request is to kindle make a series on machine learning algorithms too.
Hi Kevin I have a doubt.. I have 1 column in which time is 42368.149155 When I convert it into year, days, month, hrs, min, sec I am getting 1970-01-01, 17:22:13.453068 I read that 1970 is the default year. How can I convert it into some other year, say 2016 or any other year. Kindly help.
Hello , I have a dataset with datatime index col. and it is weekly data , do I need to set freq='W' to apply forecasting models such as Holt-Winters, I tried : df.index.freq ='W', and got this error: OverflowError: int too big to convert"" please help me to fix this. Thank you
HI Kevin, I met a problem when I read the csv file with date time. I use the following code to read the csv file, but got 2 warnings said PST and PDT can't be understood... Can you please help me solve this problem? thank you! data = pd.read_csv("datetime.csv",parse_dates = ['date/time']) Mar 3, 2019 12:16:44 AM PST UnknownTimezoneWarning: tzname PST identified but not understood. Pass `tzinfos` argument in order to correctly return a timezone-aware datetime. In a future version, this will raise an exception. category=UnknownTimezoneWarning)
Hi, I have applied the format as you had shown. But I am getting error called "AttributeError: 'DataFrame' object has no attribute 'Time'". Time is the column for my Date and time. How to solve it?
Hi - qq - I have an excel sheet that has a column that includes dates, some of the dates have errors like '4/4/4/2020' or '/1/12/2020' - - is there a way python generate a dateframe column that lists all of these errors with their corresponding row information?
This is a great video. Thank you so much! Also, I wish the pandas API reference still looked like it does in this older video. It's harder to read now.
Hello, I have 2 large datasets and want to compare time differences by seconds for instance. I want to Group-by a certain column first, and then see the time differences or duration for a certain action. Can I do this in Python
I tried to use pd.to_datetime(df.variable) with my date variable (read as an object), but I'm getting this error: OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1410-02-25 00:00:00 What am I doing wrong?
I did like this but getting error OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-21 00:00:00 , from tutorial example on web scraping of Tramp's lie
@@dataschool Great Tutorial! i also encountered this problem because of a date outlier (year 2981). Pandas would read the column as object and was unable to convert to datetime see: stackoverflow.com/questions/32888124/pandas-out-of-bounds-nanosecond-timestamp-after-offset-rollforward-plus-adding-a i added this parameter df['date'] = pd.to_datetime(df['date'], errors = 'coerce') seems to work, the column is converted and the outlier has NaN value.
I have a csv file which consist of placeid and date .I want to calculate in a particular month how many times is the particular place visited..how this could be done in python code..Please help @ Data School
Perhaps you could store the 'month' attribute from the 'date' column as a new column, then filter the DataFrame by 'placeid' column, and then check the shape of the DataFrame. This video might be helpful to you: ruclips.net/video/2AFGPdNn4FM/видео.html
Special thanks for your valuable work. Is the attribute "weekday_name" in panda version '1.1.3' now deprecated? and instead now we have 'day_name()' method?
AB['Date']=pd.to_datetime(AB.Date) i am getting error Unknown string format: TOTAL my format is dd/mm/yyyy in column Date of a data frame named AB showing dtype as object .
i have a question panda time series where all my columns are dates and i want to find the hrs spent by employee. Pls help me with your Github id so can i can post details there. please help
What if column 'Time' has a 2 words name? How do you call it on the expression ufo.Time.dt.week? I tried with ufo.ufo['Two Words'].dt.week but it doesn't work.
Thanks for your videos. I have data df['DateTime'] as '02/12/2019 11:00:00' , ...02/13/2019 23:00:00'., 02/13/2019 24:00:00' in column 48hrs of hourly data. While plot only ' 02/XX/2019 24:00:00' ( only 24:00:00 ) hrs point missing on graph. :( How to fix this? Please support me.
Basically how can i do a simple math operation like increasing 2 months to the month of any date when i do that as at the following, i'm taking an error like : 'datetime.date' objects is not writable not wirtable *** dt = pd.to_datetime('2016/9/28') dt.month = dt.month + 2
Great question! Here's how I would do it: dt = pd.to_datetime('2016/9/28') td = pd.to_timedelta('60 days') dt + td I don't think you can specify time units as months because of the ambiguity. It makes sense when you say 9/28 plus 2 months (11/28), but what about 12/31 plus 2 months (2/31?)
Hello!!! I am facing an issue regarding the DateTime problem in the data set. the problem is that when i execute the to_datetime function, there is an error pop out. the error is given below. The question is how to handle such kind of dataset. "ValueError: ('Unknown string format:', '25 - Apr-16')"" I need your help.
Hi DataSchool tk u for gr8 vid on working with dates and times. I am trying to work out how to group data for days, months and years in the same plot, e.g. bar graphs for months and different colours for the years
awesome videos. been watching quite a few now. So, I'm playing with my gpx running data. And I'm trying to convert the duration of my runs so I can plot them. But I just fail. How would you convert ints like 33:28 and 01:44:42 so it would be understood as 33 minutes and 1 hour 44 minutes and so on?
Glad you like the videos! As for your question, it seems like extracting the datetime attributes (hours, minutes, seconds) and then doing the math with those attributes would solve your problem. Hope that helps!
Hello, In My date column has the date of month missing. how do I add the date to the existing column Ex: My column is 04-1982 (which is not in date format) and I want to make it 30-04-1982.. and want to repeat for all the other sections.. please help. and how to add a date if there is no date available
The videos is with very nice explanation .I am getting the time data only in hour:min:sec format and when i convert it from object to time then it gives also the current date with time stamp. I want to fill the missing seconds values so is there any other function available ?
Python is really not intuitive when compared to R.. datetime comparison makes me split my hair . Still try to figure out how to compare a date in a dataframe ( which in datetime format) to today's date and do some action if they both match.. Any help is appreciated
Is there any simple method to calculate sum of consecutive days say employees who worked continuously for 2 weeks including weekend? Can this be done using timedelta?
Hi Kevin, lets say you have a date column containing only hour, minute and second. While changing into date format using pd.to_datetime, it added automatically years and days. How could keep only hours, minutes and seconds? Thank you.
thanks for the tutorial. is there any way to change the year column meaning change the year series. In year series - starts with 1930 and then go on till 1933, how to change this to 2013 to 2016 in the csv file ? thanks lot for the time and help ! cheers
Hi actually I had a column name time consumed which contains time in hours minutes and seconds but some of them contains only min and seconds.can u please tell me how can I get the time consumed rows within 30 sec
For anyone who wants to convert timezones, here's how you do it. First, you need to specify which timezone the date originally belongs to. In my case, it was UTC (Coordinated Universal Time). In my DataFrame (let's call it df), I have a column named "timestamp", with type datetime. I localized the column first: timestamp_utc = df["timestamp"].dt.tz_localize("UTC") Then I overwrite the timestamp column: df["timestamp"] = timestamp_utc.dt.tz_convert("Europe/Istanbul") I hope this method is correct and it helps someone!
Hi, I just tried your code but it returned all NaT values? my original column was in datetime format and UTC + x timezone. I tried to convert it to US/Eastern time. Update: Apologies, turns out the error lies in trying to convert my 'timestamp' equivalent column into date time from object and inserting errors=coerce produced the NaT values.
Thanks for the tutorials. I want to compute the difference between two dates and return the result in integer. Much like the last example you showed. can that number of days be returned as integer? thanks
Hi, I have a data-frame with the first column as year(YY) with object datatype. How do I convert the column into YYYY format. Some years are before 1970 too.
Exception has occurred: AttributeError 'DataFrame' object has no attribute 'Time' I keep getting this error when I try to use: df['date'] = pd.to_datetime(df.Time)
I've created a dataframe in python using pandas. The index used is a series of timestamp of type int64. However, for time series analysis, the index need to be type dates. Can somebody help me to do the conversion ? first few rows of the dataset is 'Elapsed time','ECG I' 'hh:mm:ss.mmm','mV' '0:00.000',-0.08 '0:00.002',-0.08 '0:00.004',-0.07 '0:00.006',-0.07 '0:00.008',-0.09 '0:00.010',-0.09 '0:00.012',-0.10 '0:00.014',-0.10 '0:00.016',-0.10 thanks in advance :)
Hi Data School, Just a quick question, I am still new at this and my apologies if this question as been answered already, how would I go about changing multiple columns to datetime? Thank you.
![I.N "HALLUCINATION" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/n5B5q1Hwt_U/mqdefault.jpg)
Starting in pandas version 0.18.1, you can create a new datetime column directly from a DataFrame, based solely on the column names! It's a useful trick, which I explain in this video: ruclips.net/video/-NbY7E9hKxk/видео.html
Hello!
I have a table with the date column. I want to group the data by month / year how do I do this?
I love you Bruh.. 😂.. No homo..thanks a lot!!!!
You've saved my job on multiple occasions sir, thank you.
That's awesome to hear! 🙌
I am an aspiring data scientist. I just found a series of your videos. Thank you for doing this for all of us. Keep doing great work!
Thanks for your kind words, and good luck to you!
I was actually going to ask you about this. I started learning Pandas thanks to your video series and I feel like I learnt a lot. Thanks for all the awesome videos.
Wow, that's really great to hear! You're very welcome!
Actually, I still need some help. I want to create a column for "year/month" from datetime column. For example, if datetime is 2016-07-06 15:56:19, I want to map "2016-07" into a column. Of course I can get first seven characters by converting it to a string, but what is the correct way to do this? I have following the following lambda function:
get_month = lambda x: '{}-{:02}'.format(x.to_datetime().year, x.to_datetime().month)
my_df["year_month"] = my_df["timestamp"].map(get_month)
For now, I do it like this but I am sure a better and more efficient way exists for the job. I'd be glad if you can help!
If your desired end result is a string (such as '2016-07'), then I think using string methods is the way to go! Perhaps something like this:
ufo.Time.dt.year.astype(str).str.cat(ufo.Time.dt.month.astype(str), sep='-')
However, there is probably an even simpler approach that I'm not thinking of...
Awesome. It really is simpler than what I was doing. Thanks!
Hi, I know its a year late, but in case you (or anyone else is interested) "pd.Series.dt.strftime" is an easy way to output dates as strings in whatever format you like.
pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.dt.strftime.html#pandas.Series.dt.strftime
You are naturally born to impart knowledge. Thanks for your videos br!
Wow, I really appreciate your kind words! 🙏
These are the first videos I look for when I have pandas questions.
Thanks!
I understood each and everything in this video and it helped me alot for my project. I just want to thank the instructor.
Great to hear!
5 years video, but the best on RUclips!!!
Thanks very much for your kind words!
I'm impressed. Simple explanations with examples. subbed and hit that bell. Thanks for the vid!
Awesome, thank you!
4:25 .weekday_name is not working, used day_name() instead (2022 update).
Thank u!
Thanks for sharing!
2022 and you're still saving us. Thanks for the excelent content
You're welcome!
I've only recently stumbled onto your videos. Very clear and concise delivery. Good job!
Thanks so much! Glad you are enjoying them.
Phenomenal description of working with dates and times in Pandas, very helpful.
I love the way you teach, easy to follow and to understand. Many Thanks.
You're welcome!
All your videos are worth watching. I have learned a lot about pandas just from your videos. Thanks a ton :)
Thanks Aarti!
I have a series of " 23:21:06 31/07/2019" format how to convert this to a time series
I'm unable to use ufo.Time.dt.weekday_name as an attribute. What might be wrong?
Try ufo.Time.dt.day_name()
Thanks for this! You explain things very clearly and concisely.
Thanks!
I really enjoyed this video and your teaching style! To the point, but well explained at a nice pace. Thank you for this :)
Thank you so much!
I've been watching your videos for a while now but never got the chance to comment on it i just want to say keep up the great work! You are just awesome!
Thanks very much for your kind words! Much appreciated :)
this is an awesome video but the to_datetime is not working for me, it keeps giving me error like "hour must be in 0..23: 10/11/2006 24:00"
I've tried everything
I have a column containing times in the format hour, minute, second (e.g. 00:24:43) and are currently an 'object' type. How can I calculate the average time for this column? I have tried converting to a pandas datetime but this throws up several different errors when I try to calculate the mean? Also, I have three separate groups (full match, T1 and T2) how can I use groupby to figure out the mean times for each group? Thanks
Thanks for your great video!
It seems pandas.Series.dt.weekday_name is removed in pandas version 0.23.0. and one can use pandas.Series.dt.day_name() instead.
Thanks for the tip. Was facing the same issue!
Thanks a lot
Thanks :)
@Data School , Mark, you are just amazing. :) You make it appear everything quite simple. My humble request is to kindle make a series on machine learning algorithms too.
Thanks for your kind words and your suggestion!
Excellent explanation .I suggested these video series to most of my friends.
Thanks so much! I really appreciate you spreading the word.
Hi Kevin
I have a doubt..
I have 1 column in which time is 42368.149155
When I convert it into year, days, month, hrs, min, sec I am getting
1970-01-01, 17:22:13.453068
I read that 1970 is the default year. How can I convert it into some other year, say 2016 or any other year. Kindly help.
You might need to adjust the "unit", see examples here: pandas.pydata.org/docs/reference/api/pandas.to_datetime.html
@@dataschool Thank You Kevin. Got it now.
You're king of this area man!!!!
Thank you!
Data School is the best of all resource available on Pandas. thanks a ton!!
Thanks!
New to Pandas, new to your channel, and soon New year =) Thanks for the videos!
You're very welcome!
Hello , I have a dataset with datatime index col. and it is weekly data , do I need to set freq='W' to apply forecasting models such as Holt-Winters, I tried : df.index.freq ='W', and got this error: OverflowError: int too big to convert"" please help me to fix this. Thank you
That bonus is what I needed. Thank you so much!
You are so welcome!
So many great videos . Absolutely guidness. thanks from GREECE !
Thank you!
HI Kevin, I met a problem when I read the csv file with date time.
I use the following code to read the csv file, but got 2 warnings said PST and PDT can't be understood... Can you please help me solve this problem? thank you!
data = pd.read_csv("datetime.csv",parse_dates = ['date/time'])
Mar 3, 2019 12:16:44 AM PST
UnknownTimezoneWarning: tzname PST identified but not understood. Pass `tzinfos` argument in order to correctly return a timezone-aware datetime. In a future version, this will raise an exception.
category=UnknownTimezoneWarning)
It's hard to say without investigating, I'm sorry!
Hi, I have applied the format as you had shown. But I am getting error called "AttributeError: 'DataFrame' object has no attribute 'Time'". Time is the column for my Date and time. How to solve it?
Hi - qq - I have an excel sheet that has a column that includes dates, some of the dates have errors like '4/4/4/2020' or '/1/12/2020' - - is there a way python generate a dateframe column that lists all of these errors with their corresponding row information?
Hi , I have a column in date/mm/yy i want to remove the yy can i do this ?
Thnks for this explanation buddy!
So clear and concise!
Thanks for your kind words!
I want to show the week of months as per the datetime columns. How can i do that? Please advise.
This is a great video. Thank you so much! Also, I wish the pandas API reference still looked like it does in this older video. It's harder to read now.
I agree...
loved the tutorial, cleared my doubts! I like how you explain so patiently.
Thank you so much in taking time to explain so nicely.
You're very welcome!
hi,
i want a function in python that identify which column have date in them??
Thanks so much. You are one of the best teachers I have ever known. Thanks so much once more you are a darling.
Wow! Thank you so much for your kind words! :)
This is EXACTLY was a looking for. I love you.
Awesome!
If I want specific dates in train data and remaining in test data how can we do tht
when i am trying to convert my data column into datetype its showing an error unknown string format
Hello, I have 2 large datasets and want to compare time differences by seconds for instance. I want to Group-by a certain column first, and then see the time differences or duration for a certain action. Can I do this in Python
I'm sure you can, but it's hard for me to say how off-hand. Sorry!
I tried to use pd.to_datetime(df.variable) with my date variable (read as an object), but I'm getting this error: OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1410-02-25 00:00:00
What am I doing wrong?
I'm not sure, sorry! I haven't seen that error before.
found it! python couldn't handle a date with year 1410
Ha! Good to know!
how can we handle ranges ( suppose salary ranges ) in a dataset? it would be great if any of you can tell.
I did like this but getting error OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-21 00:00:00 , from tutorial example on web scraping of Tramp's lie
I'm sorry, I'm not sure what the cause of this error would be.
@@dataschool Great Tutorial! i also encountered this problem because of a date outlier (year 2981). Pandas would read the column as object and was unable to convert to datetime
see: stackoverflow.com/questions/32888124/pandas-out-of-bounds-nanosecond-timestamp-after-offset-rollforward-plus-adding-a
i added this parameter
df['date'] = pd.to_datetime(df['date'], errors = 'coerce')
seems to work, the column is converted and the outlier has NaN value.
I did't get what is the difference between ufo.time* and ufo.time.dt* operations. Would You explain plz...
Hi, I have Hour column in format- 0,1,2,...22,23.How can i convert to 00:00:00,01:00:00,02:00:00...,22:00:00,23:00:00.
Thanks in Advance
Manu Madhavan
I'm not sure the easiest way to do that off-hand. Sorry! And good luck!
for h in hour_column:
datetime.time(h, 0, 0)
How can I export this "datetime64" (YYYY-MM-DD HH:MM:SS) to a CSV file with the format "DD/MM/YYYY HH:MM" ?
I have a csv file which consist of placeid and date .I want to calculate in a particular month how many times is the particular place visited..how this could be done in python code..Please help @ Data School
Perhaps you could store the 'month' attribute from the 'date' column as a new column, then filter the DataFrame by 'placeid' column, and then check the shape of the DataFrame. This video might be helpful to you: ruclips.net/video/2AFGPdNn4FM/видео.html
Excellent class! As always! Cheers!
Thank you!
I am plotting a chart in that I need to show time only, how to trim time from pandas datetime ?
Special thanks for your valuable work. Is the attribute "weekday_name" in panda version '1.1.3' now deprecated? and instead now we have 'day_name()' method?
Kevin, could we define a ts_min and ts_max, and select the events during this interval?
AB['Date']=pd.to_datetime(AB.Date)
i am getting error Unknown string format: TOTAL
my format is dd/mm/yyyy in column Date of a data frame named AB showing dtype as object .
i have a question panda time series where all my columns are dates and i want to find the hrs spent by employee. Pls help me with your Github id so can i can post details there. please help
I'm sorry, I won't be able to help. Good luck!
What if column 'Time' has a 2 words name? How do you call it on the expression ufo.Time.dt.week? I tried with ufo.ufo['Two Words'].dt.week but it doesn't work.
ufo['Two Words'].dt.week
Thanks for your videos.
I have data df['DateTime'] as '02/12/2019 11:00:00' , ...02/13/2019 23:00:00'., 02/13/2019 24:00:00' in column 48hrs of hourly data.
While plot only ' 02/XX/2019 24:00:00' ( only 24:00:00 ) hrs point missing on graph. :(
How to fix this? Please support me.
How would you convert from a column of times in UTC?
Basically how can i do a simple math operation like increasing 2 months to the month of any date
when i do that as at the following, i'm taking an error like :
'datetime.date' objects is not writable not wirtable
***
dt = pd.to_datetime('2016/9/28')
dt.month = dt.month + 2
Great question! Here's how I would do it:
dt = pd.to_datetime('2016/9/28')
td = pd.to_timedelta('60 days')
dt + td
I don't think you can specify time units as months because of the ambiguity. It makes sense when you say 9/28 plus 2 months (11/28), but what about 12/31 plus 2 months (2/31?)
Holly teacher! :D
Hello!!!
I am facing an issue regarding the DateTime problem in the data set. the problem is that when i execute the to_datetime function, there is an error pop out. the error is given below. The question is how to handle such kind of dataset.
"ValueError: ('Unknown string format:', '25 - Apr-16')""
I need your help.
You may have to define the format for to_datetime so that it understands the format. Hope that helps!
Hi DataSchool tk u for gr8 vid on working with dates and times. I am trying to work out how to group data for days, months and years in the same plot, e.g. bar graphs for months and different colours for the years
how can i calculate median for column of type datetime64 ns
awesome videos. been watching quite a few now.
So, I'm playing with my gpx running data. And I'm trying to convert the duration of my runs so I can plot them. But I just fail. How would you convert ints like 33:28 and 01:44:42 so it would be understood as 33 minutes and 1 hour 44 minutes and so on?
Glad you like the videos! As for your question, it seems like extracting the datetime attributes (hours, minutes, seconds) and then doing the math with those attributes would solve your problem. Hope that helps!
Thanks You. But a have a query
If I want to get the data between two dates
How I can do ?
I don't know the code off-hand, I'm sorry!
@@dataschool OK. Thanks . But according some of my research to have the data between a period we should set data index as time stamp type.
Hello, In My date column has the date of month missing. how do I add the date to the existing column
Ex: My column is 04-1982 (which is not in date format) and I want to make it 30-04-1982.. and want to repeat for all the other sections.. please help. and how to add a date if there is no date available
I'm sure there's a string method that can help: ruclips.net/video/bofaC0IckHo/видео.html&index=12&list=PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y
The videos is with very nice explanation .I am getting the time data only in hour:min:sec format and when i convert it from object to time then it gives also the current date with time stamp. I want to fill the missing seconds values so is there any other function available ?
I'm not sure off-hand, I'm sorry!
I'm so thankful for your tutorials
You're welcome!
Python is really not intuitive when compared to R.. datetime comparison makes me split my hair . Still try to figure out how to compare a date in a dataframe ( which in datetime format) to today's date and do some action if they both match.. Any help is appreciated
Is there any simple method to calculate sum of consecutive days say employees who worked continuously for 2 weeks including weekend? Can this be done using timedelta?
It's hard for me to say off-hand, sorry!
Bro can we use this date to split data into train and test date if yes how we can do that
Hi Kevin, lets say you have a date column containing only hour, minute and second. While changing into date format using pd.to_datetime, it added automatically years and days. How could keep only hours, minutes and seconds?
Thank you.
Great question! I can't remember off-hand, but I think you need to use a pandas timedelta object instead of a datetime object.
thanks for the tutorial.
is there any way to change the year column meaning change the year series. In year series - starts with 1930 and then go on till 1933, how to change this to 2013 to 2016 in the csv file ?
thanks lot for the time and help ! cheers
Hi actually I had a column name time consumed which contains time in hours minutes and seconds but some of them contains only min and seconds.can u please tell me how can I get the time consumed rows within 30 sec
Sorry, I won't be able to help... good luck!
Thanks for the tutorial. I have a question.
How can i find unique items under a given column as some could have been repeated?
@Data School Do you have any video on how to fill missing dates with zero (desired number) in a large csv file?
ruclips.net/video/fCMrO_VzeL8/видео.html
What about if I want to know some rows for a time interval? For example, between 2000 and 2001
I think something like this would work:
ufo[(ufo.Time.dt.year >= 2000) & (ufo.Time.dt.year < 2001)]
Excellent videos.Please consider giving tutorials on time series forecasting ( with various statistical models ) with Pandas.
Thanks for the suggestion! I'll consider it for the future.
For anyone who wants to convert timezones, here's how you do it. First, you need to specify which timezone the date originally belongs to.
In my case, it was UTC (Coordinated Universal Time). In my DataFrame (let's call it df), I have a column named "timestamp", with type datetime. I localized the column first:
timestamp_utc = df["timestamp"].dt.tz_localize("UTC")
Then I overwrite the timestamp column:
df["timestamp"] = timestamp_utc.dt.tz_convert("Europe/Istanbul")
I hope this method is correct and it helps someone!
Awesome! Thanks so much for taking the time to share your code.
Hi, I just tried your code but it returned all NaT values? my original column was in datetime format and UTC + x timezone. I tried to convert it to US/Eastern time. Update: Apologies, turns out the error lies in trying to convert my 'timestamp' equivalent column into date time from object and inserting errors=coerce produced the NaT values.
Hello, what if I want to convert two date objects variables to datetime format? I tried, but keep getting error.
Give an example?
got it. thanks for asking anyway
how Could I know the antiquity in days of a variable if this variable comes in a date format?
I don't quite understand the question, sorry!
Thanks for the tutorials. I want to compute the difference between two dates and return the result in integer. Much like the last example you showed. can that number of days be returned as integer? thanks
Actually, this already returns the result as an integer:
(ufo.Time.max() - ufo.Time.min()).days
Is there any way to separate the time (h,m,s) that's on the same column of an imported excel data frame and pass it all to seconds?
I'm sorry, I don't understand your question. Could you clarify? Thanks!
Hi, how can I convert the time (HH:MM:SS) to numerical form so that I can plot a graph of y (output) against the time in scatter plot?
You should be able to make that plot even if the time is datetime format, rather than numerical.
Hi, I have a data-frame with the first column as year(YY) with object datatype. How do I convert the column into YYYY format. Some years are before 1970 too.
Not sure off-hand, sorry!
Exception has occurred: AttributeError
'DataFrame' object has no attribute 'Time'
I keep getting this error when I try to use: df['date'] = pd.to_datetime(df.Time)
Perhaps you read the file incorrectly or have a typo somewhere?
I know im late but what if i want to change the years of the dates in my data frame, how would i do that?
Perhaps you can overwrite the year attribute of each item? There might be a better way, however.
Hello Sir! I have only year in my data not month and day. By pd.to_datetime its doesn't converting into numerical.
If I understand your question, this video might be of help instead: ruclips.net/video/V0AWyzVMf54/видео.html
wow.. you're tutorials are just so awesome !!!!!
Thank you!
How to get rows or column values over the last N days where N is a user defined parameter ? Please solve my query.
Sorry, I won't be able to help... good luck!
@@dataschool No problem. I have got the solution.
@@dataschool But I really appreciate your effort that you read every comment and reply to it. 👍🏼
You're very welcome! 😄
I've created a dataframe in python using pandas. The index used is a series of timestamp of type int64. However, for time series analysis, the index need to be type dates. Can somebody help me to do the conversion ?
first few rows of the dataset is
'Elapsed time','ECG I'
'hh:mm:ss.mmm','mV'
'0:00.000',-0.08
'0:00.002',-0.08
'0:00.004',-0.07
'0:00.006',-0.07
'0:00.008',-0.09
'0:00.010',-0.09
'0:00.012',-0.10
'0:00.014',-0.10
'0:00.016',-0.10
thanks in advance :)
Hi Data School,
Just a quick question, I am still new at this and my apologies if this question as been answered already, how would I go about changing multiple columns to datetime?
Thank you.
Hello sir, you are awesome teacher.
Great videos
Thank you very much
Thanks so much for your kind words!
What if I want to know the mean of entries for each month? Is there more ufo-sightings in summer as an example?
Is this what you are looking for?
ufo['Month'] = ufo.Time.dt.month
ufo.Month.value_counts().sort_index()
Your video is really good, we'll be really helpful, if you make some more videos on Dates and Times.
Thank you.
I cover it a bit more in this series: ruclips.net/p/PL5-da3qGB5IBITZj_dYSFqnd_15JgqwA6
thank you, Now I am planning to cover this series too.
is there a better way to encode the date-time format into ANN(keras) without using the one-hot encoder???
Not sure, sorry!