Introduction: The Problem<\/h2>\n
One issue arises when linear regression is being done on data with a single feature. Such data is often represented as a list of values (a 1-dimensional array, in most cases.) The Here we can see that a single collection of values can be interpreted in one of two ways:<\/p>\n These represent\u00a0very<\/em> different aspects of data. In the case of single-feature regression analysis, Scikit-learn’s Note<\/strong>: This post is about a nuanced aspect of data preparation for linear regression. Check out the article Simple Linear Regression<\/a><\/em> for a broader discussion or the article Predicting Stock Prices with Linear Regression in Python<\/a><\/em> for an applied tutorial.<\/p>\n Pandas commonly represent data in one of two ways: The In cases where single feature regressions are done (simple linear regression) the This error results when one attempts a call to the Note<\/strong>: The following examples are using the same 5-item collection of integers for values of both the independent and dependent variables. This is absolute nonsense and is not intended to represent a valid approach for training a regression model.<\/p>\n For the first example let’s consider a workflow using a Python We’re attempting to train the model using a 1-dimensional array. Again, this is confusing in that we aren’t specifying if this is a series of observed values for a single feature or a single observed value for multiple features. To fix this, we can add an index<\/em> value to our data as follows:<\/p>\n This simple addition of an index value in front of the values of our data lets the NumPy arrays are how Pandas represents data at the lower levels of its DataFrame and Series APIs. Knowing how to fix this issue in NumPy\u00a0generally <\/em>offers the assurance of being able to handle it in Pandas. At the very least\u2014it makes it easier to interpret the error messages! Let’s re-create our example of Python lists using a NumPy We can see that we have a 1-Dimensional array of data, showing our elements as expected. The second printout Regardless of our understanding of NumPy arrays at this point, it’s clear this approach does not work. The error message seems a bit more relevant now we’re aware that we’re using The Without assigning that to a new variable (or replacing the existing object in our case) the data will retain the same format. We can now successfully train our model without error. Given our understanding of the Pandas A If we toss our Consider the case where there may be multiple<\/em> columns present in a dataset and only certain columns are being extracted for regression analysis. Extracting these values as a There are several things to note here:<\/p>\n If we try to train our model as We can see both methods will properly reshape our data such that scikit-learn’s This article was motivated by a continued lack of familiarity with the The use of double bracket vs. single bracket notation in Pandas\u2014resulting in either a Pandas, NumPy, and Scikit-Learn are three Python libraries used for linear regression. Scitkit-learn’s LinearRegression class is able to easily instantiate, be trained, and be applied in a few lines of code. Depending on how data is loaded, accessed, and passed around, there can be some issues that will cause errors. These errors can be addressed […]<\/p>\n","protected":false},"author":2,"featured_media":1189,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[655,10],"tags":[68,211,239,456,238,198,210,143,455,200,440],"acf":[],"_links":{"self":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/1186"}],"collection":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/comments?post=1186"}],"version-history":[{"count":7,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/1186\/revisions"}],"predecessor-version":[{"id":3227,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/1186\/revisions\/3227"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media\/1189"}],"wp:attachment":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media?parent=1186"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/categories?post=1186"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/tags?post=1186"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}LinearRegression<\/code> model doesn’t know if this is a series of observed values for a single feature<\/strong> or a single observed value for multiple features<\/strong>. Let’s try to visualize the issue:<\/p>\n![\"shape](\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1186\/shape-comparisons.jpg\" "\\"shape-comparisons\\"")
<\/a>\n
LinearRegresion<\/code> class needs to be explicitly told that a series of data represents a series of observed values for a single feature and not the other way around. Fortunately, this can be done fairly easily in one of several ways. Before we get into how to solve this issue let’s consider first how it might arise.<\/p>\nPandas DataFrames, Series, and NumPy Arrays<\/h2>\n
DataFrame<\/code> objects or Series<\/code> objects. Without diving too deeply; DataFrames<\/code> are like spreadsheets\u2014they represent rows and columns of data. DataFrame<\/code> objects can have many rows and many columns. Consider the following illustration:<\/p>\n<\/a> Series<\/code> objects are like a single column from spreadsheets\u2014they can have many rows but only a single column. DataFrames<\/code> are essentially a collection of Series<\/code> objects that are given an index value by Pandas.\u00a0 Under the hood, the data are represented as NumPy Array<\/code> objects. That’ll be important to know in just a minute.<\/p>\nScikit-Learn & LinearRegression<\/h2>\n
scikit-learn<\/code> library is a powerful set of tools for machine learning in Python. Among its many utilities is the LinearRegression<\/code> class. This class makes developing a linear model, training it, and using it to make predictions extremely simple.<\/p>\nLinearRegression<\/code> class needs to be instructed this is a series of overserved values for a single variable. Otherwise, the following error message is likely to be thrown:<\/p>\nReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.\r\n<\/pre>\n
LinearRegression<\/code> class’ fit()<\/code> method. This error can arise for any number of reasons depending on one’s data workflow. If, for example, one is using Pandas DataFrame<\/code> objects, it’s generally an issue of indexing syntax when extracting one column from several. When using numpy arrays<\/code>, it’s generally an issue of not having an index value. Let’s look at some examples for each.<\/p>\nNative Python Lists<\/h2>\n
list<\/code> as our starting point. From there, we’ll pass that as an argument for both the independent<\/a> and dependent variables<\/a> to the LinearRegression<\/code> class.<\/p>\nfrom sklearn.linear_model import LinearRegression\r\n\r\n# Make up some data\r\ndata = [1, 2, 3, 4, 5]\r\n\r\n# Instantiate new Regression model\r\nregr = LinearRegression()\r\n\r\n# Train the model\r\nregr.fit(data, data) # error here\r\n\r\n# Result\r\nReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.\r\n<\/pre>\n
from sklearn.linear_model import LinearRegression\r\n\r\n# Add index value\r\ndata = [[i, x] for i, x in enumerate(data)]\r\n\r\n# View new collection\r\nprint(data)\r\n\r\n# Result - a 2D array\r\n[[0, 1], [1, 2], [2, 3], [3, 4], [4, 5]]\r\n\r\n# Instantiate new Regression model\r\nregr = LinearRegression()\r\n\r\n# Train the model\r\nregr.fit(data, data)\r\n\r\n# no error<\/pre>\n
LinearRegression<\/code> model know our data is for a single feature. This is the equivalent of listening to scikit-learn<\/code>‘s error message “Reshape your data either using array.reshape(-1, 1) if your data has a single feature.” Now that we have an idea of what’s going on let’s take a look at how NumPy handles this.<\/p>\nNumPy Arrays<\/h2>\n
array<\/code> instead of a Python list<\/code>.<\/p>\nimport numpy as np\r\nfrom sklearn.linear_model import LinearRegression\r\n\r\n# Create our data as an array object\r\ndata = np.array([1, 2, 3, 4, 5])\r\n\r\n# Examine the data\r\nprint(data)\r\nprint(data.shape)\r\n\r\n# Result\r\n[1 2 3 4 5]\r\n(5,)\r\n\r\n# Try to train the model\r\nregr = LinearRegression()\r\nregr.fit(data, data)\r\n\r\n# Result\r\nReshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.<\/pre>\n
(5,)<\/code> is a little mysterious. Without diving too deeply into NumPy data structures, this essentially says a collection of 5 elements without information on how that data is organized. For a better idea of how NumPy represents arrays, indexes them, and all the implications I suggest checking out this incredible<\/em> post on StackOverflow.<\/a><\/p>\nnp.array<\/code> objects\u2014which are what scikit-learn<\/code>‘s error message refers to with “array.” That means we can use the np.array.reshape()<\/code> method here to fix our data as suggested:<\/p>\nimport numpy as np\r\nfrom sklearn.linear_model import LinearRegression\r\n\r\n# Create data the same way\r\ndata = np.array([1, 2, 3, 4, 5])\r\nprint(data.shape)\r\n(5,)\r\n\r\n# Add an index value by \"reshaping\" data\r\ndata.reshape(-1, 1)\r\nprint(data.shape)\r\n(5,) # need new reference to return value\r\n\r\n# Add an index value by \"reshaping\" data\r\n# and assigning to new object\r\ndata = data.reshape(-1, 1)\r\nprint(data.shape)\r\n(5, 1)\r\n\r\n# Train the model again\r\nregr = LinearRegression()\r\nregr.fit(data, data)<\/pre>\n
numpy.array.reshape()<\/code> method uses the value of -1<\/code> to mean “automatically determine a value by counting the number of elements.” This indexes our data sequentially starting from the first member and moving to the last. It’s also important to note that the reshape()<\/code> method returns a\u00a0copy\u00a0<\/em>of the data object and does not modify the existing object.<\/p>\nnumpy.array<\/code> data structure we can now understand how to approach the issue when using Pandas DataFrames<\/code>.<\/p>\nPandas DataFrames<\/h2>\n
DataFrames<\/code> are a collection of Pandas Series<\/code> objects. The DataFrame<\/code> is an n-dimensional object (the “n” being the number of Series<\/code> contained) and the Series<\/code> is a 1-dimensional object.<\/p>\nDataFrame<\/code> objects are indexed such that a DataFrame<\/code> containing a single Series<\/code> object is considered a 2-dimensional array, where the first dimension is an index value. A Series<\/code> object is considered a 1-dimensional array.<\/p>\nSeries<\/code> object is still technically\u00a0<\/em>a 2D array, where the first dimension is an index and the second the values. This is clear when one prints a representation to stdout<\/code>. However, it’s interpreted as a 1D array such that scikit-learn’s LinearRegression<\/code> class will regard it as such. This is where things get a little unclear so let’s consider some examples:<\/p>\nimport pandas as pd\r\n\r\n# Create a dataframe with our values labeled as \"observed_values\"\r\ndf = pd.DataFrame({'observed_values': [1, 2, 3, 4, 5]})\r\nprint(df)\r\nprint(type(df))\r\nprint(df.shape)\r\nprint(df.index)\r\n\r\n# The Dataframe\r\n observed_values\r\n0 1\r\n1 2\r\n2 3\r\n3 4\r\n4 5\r\n\r\n# The type\r\n<class 'pandas.core.frame.DataFrame'>\r\n\r\n# Its dimensions\r\n(5, 1)\r\n\r\n# Its index\r\nRangeIndex(start=0, stop=5, step=1)<\/pre>\nDataFrame<\/code> object into the LinearRegression.fit()<\/code> method we’ll not get any errors. This is because our data is a single column with a valid index interpreted as such (evident by the df.shape<\/code> call).<\/p>\nSeries<\/code> is where things can go awry. Consider the following:<\/p>\nimport pandas as pd\r\n\r\n# Create our dataframe object\r\ndf = pd.DataFrame({'observed_values': [1, 2, 3, 4, 5]})\r\n\r\n# Extract a single column for use\r\ndata = df['observed_values']\r\n\r\n# Print some info\r\nprint(data)\r\nprint(type(data ))\r\nprint(data.shape)\r\nprint(data.index)\r\n\r\n# The representation\r\n0 1\r\n1 2\r\n2 3\r\n3 4\r\n4 5\r\nName: observed_values, dtype: int64\r\n\r\n# Our object type\r\n<class 'pandas.core.series.Series'>\r\n\r\n# Our object shape\r\n(5,)\r\n\r\n# Our object Index\r\nRangeIndex(start=0, stop=5, step=1)<\/pre>\n\n
Series<\/code> object, not a DataFrame<\/code><\/li>\nSeries<\/code>‘ shape anymore.<\/li>\n<\/ol>\nregr.fit(data, data)<\/code> again we’ll run into the same error message as before indicating our data is of a confusing shape. There are two approaches to remedy this situation\u2014a proactive<\/em> approach and a reactive<\/em> approach. Let’s consider both:<\/p>\n# Approach One - Extract Series as DataFrame\r\ndata = df[['observed_values']]\r\n\r\n# Print Summaries\r\nprint(data)\r\nprint(type(data))\r\nprint(data.shape)\r\nprint(data.index)\r\n\r\n# result\r\n observed_values\r\n0 1\r\n1 2\r\n2 3\r\n3 4\r\n4 5\r\n<class 'pandas.core.frame.DataFrame'>\r\n(5, 1)\r\nRangeIndex(start=0, stop=5, step=1)\r\n\r\n# Aproach Two - use the np.reshape() method\r\ndata = df['observed_values'].values.reshape(-1, 1)\r\n\r\n# print summary \r\nprint(data)\r\nprint(type(data))\r\nprint(data.shape)\r\n# print(data.index) - note: np.array doesn't have an index method\r\n\r\n# Result\r\n[[1]\r\n [2]\r\n [3]\r\n [4]\r\n [5]]\r\n<class 'numpy.ndarray'>\r\n(5, 1)<\/pre>\n
LinearRegrssion()<\/code> model will recognize it as being a series of observed values for a single feature.<\/p>\nFinal Thoughts<\/h2>\n
DataFrame<\/code> API, the implications of indexing syntaxes, and its widespread integration among Python’s most popular data science packages.<\/p>\nSeries<\/code> object or DataFrame<\/code> object\u2014was a real<\/em> point of confusion for me. The official documentation<\/a> for indexing and accessing data with Pandas is helpful\u2014I just continually forgot the RTFM<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"