Date and Time#

With pandas you can create Series with date and time information. In the following we will show common operations with date data.

Note:

pandas supports dates stored in UTC values using the datetime64[ns] datatype. Local times from a single time zone are also supported. Multiple time zones are supported by a pandas.Timestamp object. If you need to handle times from multiple time zones, I would probably split the data by time zone and use a separate DataFrame or Series for each time zone.

See also:

Loading UTC time data#

[1]:

import pandas as pd


dt = pd.date_range("2022-03-27", periods=6, freq="H")

dt

[1]:

DatetimeIndex(['2022-03-27 00:00:00', '2022-03-27 01:00:00',
               '2022-03-27 02:00:00', '2022-03-27 03:00:00',
               '2022-03-27 04:00:00', '2022-03-27 05:00:00'],
              dtype='datetime64[ns]', freq='H')

[2]:

utc = pd.to_datetime(dt, utc=True)

utc

[2]:

DatetimeIndex(['2022-03-27 00:00:00+00:00', '2022-03-27 01:00:00+00:00',
               '2022-03-27 02:00:00+00:00', '2022-03-27 03:00:00+00:00',
               '2022-03-27 04:00:00+00:00', '2022-03-27 05:00:00+00:00'],
              dtype='datetime64[ns, UTC]', freq='H')

Note:

The type of the result dtype='datetime64[ns, UTC]' indicates that the data is stored as UTC.

Let’s convert this series to the time zone Europe/Berlin:

[3]:

utc.tz_convert("Europe/Berlin")

[3]:

DatetimeIndex(['2022-03-27 01:00:00+01:00', '2022-03-27 03:00:00+02:00',
               '2022-03-27 04:00:00+02:00', '2022-03-27 05:00:00+02:00',
               '2022-03-27 06:00:00+02:00', '2022-03-27 07:00:00+02:00'],
              dtype='datetime64[ns, Europe/Berlin]', freq='H')

Conversion of local time to UTC#

[4]:

local = utc.tz_convert("Europe/Berlin")

local.tz_convert("UTC")

[4]:

DatetimeIndex(['2022-03-27 00:00:00+00:00', '2022-03-27 01:00:00+00:00',
               '2022-03-27 02:00:00+00:00', '2022-03-27 03:00:00+00:00',
               '2022-03-27 04:00:00+00:00', '2022-03-27 05:00:00+00:00'],
              dtype='datetime64[ns, UTC]', freq='H')

Conversion to Unix time#

If you have a Series with UTC or local time information, you can use this code to determine the seconds according to Unix time:

[5]:

uts = pd.to_datetime(dt).view(int) / 10**9

uts

[5]:

array([1.6483392e+09, 1.6483428e+09, 1.6483464e+09, 1.6483500e+09,
       1.6483536e+09, 1.6483572e+09])

To load the Unix time in UTC, you can proceed as follows:

[6]:

(pd.to_datetime(uts, unit="s").tz_localize("UTC"))

[6]:

DatetimeIndex(['2022-03-27 00:00:00+00:00', '2022-03-27 01:00:00+00:00',
               '2022-03-27 02:00:00+00:00', '2022-03-27 03:00:00+00:00',
               '2022-03-27 04:00:00+00:00', '2022-03-27 05:00:00+00:00'],
              dtype='datetime64[ns, UTC]', freq=None)

Manipulation of dates#

Convert to strings#

With pandas.DatetimeIndex you have some possibilities to convert date and time into strings, for example into the name of the weekday:

[7]:

local.day_name(locale="en_GB.UTF-8")

[7]:

Index(['Sunday', 'Sunday', 'Sunday', 'Sunday', 'Sunday', 'Sunday'], dtype='object')

You can find out which locale is available to you with locale -a:

[8]:

!locale -a

en_NZ
nl_NL.UTF-8
pt_BR.UTF-8
fr_CH.ISO8859-15
eu_ES.ISO8859-15
en_US.US-ASCII
af_ZA
bg_BG
cs_CZ.UTF-8
fi_FI
zh_CN.UTF-8
eu_ES
sk_SK.ISO8859-2
nl_BE
fr_BE
sk_SK
en_US.UTF-8
en_NZ.ISO8859-1
de_CH
sk_SK.UTF-8
de_DE.UTF-8
am_ET.UTF-8
zh_HK
be_BY.UTF-8
uk_UA
pt_PT.ISO8859-1
en_AU.US-ASCII
kk_KZ.PT154
en_US
nl_BE.ISO8859-15
de_AT.ISO8859-1
hr_HR.ISO8859-2
fr_FR.ISO8859-1
af_ZA.UTF-8
am_ET
fi_FI.ISO8859-1
ro_RO.UTF-8
af_ZA.ISO8859-15
en_NZ.UTF-8
fi_FI.UTF-8
hr_HR.UTF-8
da_DK.UTF-8
ca_ES.ISO8859-1
en_AU.ISO8859-15
ro_RO.ISO8859-2
de_AT.UTF-8
pt_PT.ISO8859-15
sv_SE
fr_CA.ISO8859-1
fr_BE.ISO8859-1
en_US.ISO8859-15
it_CH.ISO8859-1
en_NZ.ISO8859-15
en_AU.UTF-8
de_AT.ISO8859-15
af_ZA.ISO8859-1
hu_HU.UTF-8
et_EE.UTF-8
he_IL.UTF-8
uk_UA.KOI8-U
be_BY
kk_KZ
hu_HU.ISO8859-2
it_CH
pt_BR
ko_KR
it_IT
fr_BE.UTF-8
ru_RU.ISO8859-5
zh_TW
zh_CN.GB2312
no_NO.ISO8859-15
de_DE.ISO8859-15
en_CA
fr_CH.UTF-8
sl_SI.UTF-8
uk_UA.ISO8859-5
pt_PT
hr_HR
cs_CZ
fr_CH
he_IL
zh_CN.GBK
zh_CN.GB18030
fr_CA
pl_PL.UTF-8
ja_JP.SJIS
sr_YU.ISO8859-5
be_BY.CP1251
sr_YU.ISO8859-2
sv_SE.UTF-8
sr_YU.UTF-8
de_CH.UTF-8
sl_SI
pt_PT.UTF-8
ro_RO
en_NZ.US-ASCII
ja_JP
zh_CN
fr_CH.ISO8859-1
ko_KR.eucKR
be_BY.ISO8859-5
nl_NL.ISO8859-15
en_GB.ISO8859-1
en_CA.US-ASCII
is_IS.ISO8859-1
ru_RU.CP866
nl_NL
fr_CA.ISO8859-15
sv_SE.ISO8859-15
hy_AM
en_CA.ISO8859-15
en_US.ISO8859-1
zh_TW.Big5
ca_ES.UTF-8
ru_RU.CP1251
en_GB.UTF-8
en_GB.US-ASCII
ru_RU.UTF-8
eu_ES.UTF-8
es_ES.ISO8859-1
hu_HU
el_GR.ISO8859-7
en_AU
it_CH.UTF-8
en_GB
sl_SI.ISO8859-2
ru_RU.KOI8-R
nl_BE.UTF-8
et_EE
fr_FR.ISO8859-15
cs_CZ.ISO8859-2
lt_LT.UTF-8
pl_PL.ISO8859-2
fr_BE.ISO8859-15
is_IS.UTF-8
tr_TR.ISO8859-9
da_DK.ISO8859-1
lt_LT.ISO8859-4
lt_LT.ISO8859-13
zh_TW.UTF-8
bg_BG.CP1251
el_GR.UTF-8
be_BY.CP1131
da_DK.ISO8859-15
is_IS.ISO8859-15
no_NO.ISO8859-1
nl_NL.ISO8859-1
nl_BE.ISO8859-1
sv_SE.ISO8859-1
pt_BR.ISO8859-1
zh_CN.eucCN
it_IT.UTF-8
en_CA.UTF-8
uk_UA.UTF-8
de_CH.ISO8859-15
de_DE.ISO8859-1
ca_ES
sr_YU
hy_AM.ARMSCII-8
ru_RU
zh_HK.UTF-8
eu_ES.ISO8859-1
is_IS
bg_BG.UTF-8
ja_JP.UTF-8
it_CH.ISO8859-15
fr_FR.UTF-8
ko_KR.UTF-8
et_EE.ISO8859-15
kk_KZ.UTF-8
ca_ES.ISO8859-15
en_IE.UTF-8
es_ES
de_CH.ISO8859-1
en_CA.ISO8859-1
es_ES.ISO8859-15
en_AU.ISO8859-1
el_GR
da_DK
no_NO
it_IT.ISO8859-1
en_IE
zh_HK.Big5HKSCS
hi_IN.ISCII-DEV
ja_JP.eucJP
it_IT.ISO8859-15
pl_PL
ko_KR.CP949
fr_CA.UTF-8
fi_FI.ISO8859-15
en_GB.ISO8859-15
fr_FR
hy_AM.UTF-8
no_NO.UTF-8
es_ES.UTF-8
de_AT
tr_TR.UTF-8
de_DE
lt_LT
tr_TR
C
POSIX

Other attributes of DatetimeIndex that can be used to convert date and time into strings are:

Attribute

Description

year

the year as datetime.

month

the month as January 1 and December 12

day

the day of the datetime

hour

the hours of the datetime

minute

the minutes of the datetime

seconds

the seconds of the ‘datetime

microsecond

the microseconds of the datetime.

nanosecond

the nanoseconds of datetime

date

returns a NumPy array of Python datetime.date objects

time

returns a NumPy array of datetime.time objects

timetz

returns a NumPy array of datetime.time objects with timezone information

dayofyear, day_of_year

the ordinal day of the year

dayofweek

the day of the week with Monday (0) and Sunday (6)

day_of_week

the day of the week with Monday (0) and Sunday (6)

weekday

the day of the week with Monday (0) and Sunday (6)

quarter

returns the quarter of the year

tz

returns the time zone

freq

returns the frequency object if it is set, otherwise None

freqstr

returns the frequency object as a string if it is set, otherwise None

is_month_start

indicates if the date is the first day of the month

is_month_end

indicates whether the date is the last day of the month

is_quarter_start

indicates whether the date is the first day of a quarter

is_quarter_end

shows if the date is the last day of a quarter

is_year_start

indicates whether the date is the first day of a year

is_year_end

indicates whether the date is the last day of a year

is_leap_year

Boolean indicator if the date falls in a leap year

inferred_freq

tries to return a string representing a frequency determined by infer_freq

However, there are also some methods with which you can convert the DatetimeIndex into strings, for example strftime:

[9]:

local.strftime("%d.%m.%Y")

[9]:

Index(['27.03.2022', '27.03.2022', '27.03.2022', '27.03.2022', '27.03.2022',
       '27.03.2022'],
      dtype='object')

Note:

In strftime() and strptime() Format Codes you get an overview of the different formatting possibilities of strftime.

Other methods are:

Method

Description

normalize

converts times to midnight

strftime

converts to index using the specified date format

snap

snaps the timestamp to the next occurring frequency

tz_convert

convert a tz capable datetime array/index from one time zone to another

tz_localize

localises tz-naive datetime array/index into tz-compatible datetime array/index

round

rounds the data up to the nearest specified frequency

floor

rounds the data sown to the specified frequency

ceil

round the data to the specified frequency

to_period

converts the data to a PeriodArray/Index at a given frequency

to_perioddelta

calculates TimedeltaArray of the difference between the index values and the index converted to PeriodArray at the specified frequency

to_pydatetime

returns Datetime array/index as ndarray object of datetime.datetime objects

to_series

creates a series with index and values corresponding to index keys; useful with map for returning an indexer

to_frame

creates a DataFrame with a column containing the index

month_name

returns the month names of the DateTimeIndex with the specified locale

day_name

returns the day names of the DateTimeIndex with the specified locale

mean

returns the mean value of the array

std

returns the standard deviation of the sample across the requested axis