Filtering Data For Multiple Years By Date Range Given By Months And Days In Pandas Dataframe

I have a data frame, df, in the following format for multiple years (6hourly). I need to filter the dates, for every year, just for the period 30th November to 30th March. D

Solution 1:

imagine that you want to select only datas from November 30 to March 30:

then you can divide between months that you select completely and months of which you only select a few days:

1.you select completely: December January and February.

Dcember_Jan_Feb=~df['Dates'].dt.month.between(3,11)

you select on November 30, and you also select from March 1 to March 30 (March has 31 days)

Nov_30=df['Dates'].dt.month.eq(11)&df['Dates'].dt.day.between(30,30)
March_1_to_30=df['Dates'].dt.month.eq(3)&df['Dates'].dt.day.between(1,30)

Finally, apply your filter:

df_filtered=df[Dcember_Jan_Feb|Nov_30|March_1_to_30]

Solution 2:

here is a trick way of doing it

df = pd.DataFrame([('11178', '2018-10-27 12:00:00', '-1', '-3'), ('11179', '2018-03-30 18:00:00', '-2', '2'), ('11180', '2018-10-28 00:00:00', '1', '8'), ('11181', '2018-10-28 06:00:00', '0.1', '-0.2'), ('11182', '2018-10-28 12:00:00', '0.2', '-0.1'), ('11183', '2018-10-28 18:00:00', '0.2', '0.03'), ('11184', '2018-4-29 00:00:00', '0.3', '0.07'), ('11185', '2018-9-29 06:00:00', '0.2', '-0.2'), ('11186', '2018-10-29 12:00:00', '0.05', '-0.4'), ('11187', '2018-10-29 18:00:00', '0.02', '-0.5'), ('11188', '2018-10-30 00:00:00', '0.02', '-0.6'), ('11189', '2018-10-30 06:00:00', '-0.05', '-0.7')], columns=('ID', 'Dates', 'Col1', 'Col2'))
df = df.set_index("ID")
df.Dates = pd.to_datetime(df.Dates)

df[~(df.Dates.dt.month*100 + df.Dates.dt.day).between(330, 930, inclusive=False)]

Description

• 100*month + day will convert date as 3 digit number like 330 will be march 30 and 930 will be september 30
• you want to exclude dates between 330 and 930 so we use not operator to dates within it