{"id":930,"date":"2021-07-01T15:32:46","date_gmt":"2021-07-01T19:32:46","guid":{"rendered":"https:\/\/www.alpharithms.com\/?p=930"},"modified":"2022-06-16T15:24:07","modified_gmt":"2022-06-16T19:24:07","slug":"pandas-time-series-select-sub-set-range","status":"publish","type":"post","link":"https:\/\/www.alpharithms.com\/pandas-time-series-select-sub-set-range-463215\/","title":{"rendered":"Pandas: Select a Sub-Set Range of Dates in Time Series Data"},"content":{"rendered":"

Time Series data is used in all types of applications ranging from weather monitoring to stock trading. Pandas DataFrame objects can natively index a Time Series data and provide performant analysis.<\/p>\n

A common issue is that one needs only analyze a subset of dates from a larger Time Series data. In this quick example, we’ll use the .loc<\/code> method of Pandas DataFrames<\/code> to select a range of dates from Time Series data.<\/p>\n

Selecting a Range of Time Series Dates<\/h2>\n

Before we select a range of dates from our Time Series in pandas let’s go over some quick assumptions. First and foremost I’m assuming you have loaded data into a DataFrame<\/code> per standard practices outlined in the official documentation<\/a>. With that assumption, we also are making the following assumptions:<\/p>\n

    \n
  1. We have data<\/li>\n
  2. The data is loaded into memory as a DataFrame<\/code> object<\/li>\n
  3. The data has an associated sequence of dates<\/li>\n<\/ol>\n

    Just so everyone is on the same page, let’s take a look at our data. Below is a Time Series data containing the Adjusted Close price for Bitcoin from the dates 2014-09-17<\/code> to 2021-06-26<\/code>. For the curious, that reflects a relative price increase<\/a> of 6668.4832<\/strong>%.<\/p>\n

    # Inspect our data\r\n>>> df\r\n\r\nDate\r\n2014-09-17      457.334015\r\n2014-09-18      424.440002\r\n2014-09-19      394.795990\r\n2014-09-20      408.903992\r\n2014-09-21      398.821014\r\n                  ...     \r\n2021-06-22    32505.660156\r\n2021-06-23    33723.027344\r\n2021-06-24    34662.437500\r\n2021-06-25    31637.779297\r\n2021-06-26    30954.576172\r\nName: Adj Close, Length: 2475, dtype: float64<\/pre>\n
    Check\/Set Index Value<\/h2>\n
    The first thing we need to do is to ensure our data is properly indexed by the column containing the date values. This value is often set during instantiation of the DataFrame<\/code> object by using the index_col<\/code> argument. It can be set to the index value of the Date column or verbose string name (e.g. “Date”.) We double-check this as follows:<\/p>\n
    # Check the current index of our data\r\n>>> df.index\r\n\r\nDatetimeIndex(['2014-09-17', '2014-09-18', '2014-09-19', '2014-09-20',\r\n               '2014-09-21', '2014-09-22', '2014-09-23', '2014-09-24',\r\n               '2014-09-25', '2014-09-26',\r\n               ...\r\n               '2021-06-17', '2021-06-18', '2021-06-19', '2021-06-20',\r\n               '2021-06-21', '2021-06-22', '2021-06-23', '2021-06-24',\r\n               '2021-06-25', '2021-06-26'],\r\n              dtype='datetime64[ns]', name='Date', length=2475, freq=None)<\/pre>\n
    Optional: Visualize the Data<\/h2>\n
    Just for kicks, let’s visualize this Time Series as a line plot quickly:<\/p>\n
    \"time
    A visual representation of our Time Series data<\/figcaption><\/figure>\n
    While not strictly necessary for our example here I find plots helpful in assessing which ranges of data one might want to further analyze. In those cases, the period between 2017 and 2019 looks pretty interesting. Let’s isolate that range of Time Series Data in Pandas.<\/p>\n
    Slice Indexes via .loc Method<\/h2>\n
    There are a number of ways to select ranges from Time Series data in pandas. I find the easiest to be using the .loc operator. It is highly optimized and offers a friendly syntactic use. Let’s slice into our Time Series:<\/p>\n
    # Select range\r\n>>> data.loc['2017-06-01':'2019-01-01']\r\n\r\nDate\r\n2017-06-01    2407.879883\r\n2017-06-02    2488.550049\r\n2017-06-03    2515.350098\r\n2017-06-04    2511.810059\r\n2017-06-05    2686.810059\r\n                 ...     \r\n2018-12-28    3923.918701\r\n2018-12-29    3820.408691\r\n2018-12-30    3865.952637\r\n2018-12-31    3742.700439\r\n2019-01-01    3843.520020\r\nName: Adj Close, Length: 580, dtype: float64<\/pre>\n
    Our DataFrame Time Series object now contains data restricted to the range of June 1st, 2017 through January 1st, 2019 (the interesting part of the chart above.) Let’s take another look at this chart to confirm visually.<\/p>\n
    \"sliced
    Plot of our Time Series with the selected range from mid 2017 – 2019<\/figcaption><\/figure>\n
    Final Thoughts<\/h2>\n
    This quick example demonstrates how Pandas provides an easy sub-selection of Time Series data by a range of dates. Here we’ve used what Pandas refers to as Partial String Indexing<\/a> to achieve our desired results. Alternative approaches include using the mask feature<\/a>. One parting word of advice is that, for those loading Time Series data via CSV files, making use of the parse_dates<\/code> and index_col<\/code> arguments will save much heartache.<\/p>\n","protected":false},"excerpt":{"rendered":"
    Time Series data is used in all types of applications ranging from weather monitoring to stock trading. Pandas DataFrame objects can natively index a Time Series data and provide performant analysis. A common issue is that one needs only analyze a subset of dates from a larger Time Series data. In this quick example, we’ll […]<\/p>\n","protected":false},"author":2,"featured_media":935,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[186,655],"tags":[192,380,382,191,210,383,212,381],"acf":[],"_links":{"self":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/930"}],"collection":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/comments?post=930"}],"version-history":[{"count":1,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/930\/revisions"}],"predecessor-version":[{"id":934,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/930\/revisions\/934"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media\/935"}],"wp:attachment":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media?parent=930"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/categories?post=930"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/tags?post=930"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}