{"id":1089,"date":"2021-07-13T16:25:36","date_gmt":"2021-07-13T20:25:36","guid":{"rendered":"https:\/\/www.alpharithms.com\/?p=1089"},"modified":"2022-06-16T15:24:05","modified_gmt":"2022-06-16T19:24:05","slug":"covariance","status":"publish","type":"post","link":"https:\/\/www.alpharithms.com\/covariance-362516\/","title":{"rendered":"Covariance: Finding Direction Among Variable Relationships"},"content":{"rendered":"<p>Covariance is a statistical method for describing how two variables change in relationship to one another. Its measure is used to describe directional correlation such that the change in observed value for one variable is reflected by a change in another. Covariance is integral to modern statistics but often only as accompanying support within more comprehensive methods.<\/p>\n<p>Covariance has applications in finance\u2014where it can help with <strong>portfolio management<\/strong> by reducing risk (Bartz, 2013), <strong>machine learning<\/strong>\u2014where it can help <strong>dimensionality reduction<\/strong> (Jang, 2007), and in clinical study design\u2014where it can help initial stages of <strong>identifying causal relationships<\/strong> (Schlotz, 2008).<\/p>\n<h2>Highlights<\/h2>\n<ul>\n<li>Covariance describes the relationship between two variables.<\/li>\n<li>It provides a measure of\u00a0<em>direction<\/em> but not the <em>magnitude.<\/em><\/li>\n<li>It is only used to measure the relationship between <em>two<\/em> variables and requires an equal number of serial observations for each.<\/li>\n<li>Applications include portfolio management, clinical study design, and forecasting stock prices.<\/li>\n<li>Covariance is integral to many machine learning algorithms and a key aspect of correlation analysis and regression analysis.<\/li>\n<li>NumPy, Pandas and native Python code can be used to quickly and efficiently calculate the covariance.<\/li>\n<\/ul>\n<h2>Introduction<\/h2>\n<p>Covariance is the formal mathematical expression of how one variable changes in response to another. A pair of variables are considered to have <strong>positive covariance<\/strong> if an increase in observed value for one variable is reflected by an increase in observed value for the other variable. Likewise, <strong>negative covariance<\/strong> is noted when the observed value of one variable decreases in response to the decrease in the other.<\/p>\n<p>Simple covariance calculations are used in cases where two variables are being considered, such as with <strong>simple linear regression<\/strong>. These reflect an <a href=\"https:\/\/www.alpharithms.com\/term\/independent-variable\/\">independent variable (predictor)<\/a> and a <a href=\"https:\/\/www.alpharithms.com\/term\/dependent-variable\/\">dependent variable (response.)<\/a> Covariance measures\u00a0<em>series<\/em> data where an identical number of observed values are present for each variable.<\/p>\n<p>Below are two charts illustrating the movements of two sets of hypothetical asset pairs.<\/p>\n<figure id=\"attachment_1125\" aria-describedby=\"caption-attachment-1125\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-positive-vs-negative.jpg\"><img fetchpriority=\"high\" decoding=\"async\" class=\"size-full wp-image-1125\" title=\"covariance-positive-vs-negative\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-positive-vs-negative.jpg\" alt=\"covariance positive vs negative\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-positive-vs-negative.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-positive-vs-negative-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-positive-vs-negative-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-positive-vs-negative-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-positive-vs-negative-780x439.jpg 780w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1125\" class=\"wp-caption-text\">Depiction of observed values among variables exhibiting positive and negative covariances. (click to enlarge)<\/figcaption><\/figure>\n<p>The left chart depicts assets with a strong negative covariance such as an increase in observed value for one variable is reflected by a decrease in observed value for the other. The right chart depicts two assets with strong positive covariance such that an increase in observed value for one variable is reflected by an increase in observed value for the other.<\/p>\n<p>Covariance can be used to help analyze stock performance, better manage portfolio risk, and identify features for machine learning models. We&#8217;ll cover each of these applications in more detail a bit later. For now, let&#8217;s start walking through how to calculate the covariance of two series of observed values.<\/p>\n<h2>Calculating the Covariance<\/h2>\n<figure id=\"attachment_1172\" aria-describedby=\"caption-attachment-1172\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram.jpg\"><img decoding=\"async\" class=\"border-image wp-image-1172 size-large\" title=\"covariance-equation-diagram\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram-1024x576.jpg\" alt=\"covariance equation diagram\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram-1024x576.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram-780x439.jpg 780w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1156\/covariance-equation-diagram.jpg 1366w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1172\" class=\"wp-caption-text\">The Covariance of two variables can be calculated by summing the the difference of each observed value and the sample mean divided by the total sample population minus one. (click to enlarge)<\/figcaption><\/figure>\n<p>To better define covariance let&#8217;s walk through a stepwise process of its calculation. To get started, let&#8217;s define some data that will be used throughout the process:<\/p>\n<table id=\"covariance-sample-data\" class=\"centered-text\">\n<thead>\n<tr>\n<th>Observation<br \/>\nNumber<\/th>\n<th>Predictor<br \/>\n(X)<\/th>\n<th>Response<br \/>\n(Y)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>1<\/td>\n<td>10<\/td>\n<td>3.15<\/td>\n<\/tr>\n<tr>\n<td>2<\/td>\n<td>11<\/td>\n<td>3.47<\/td>\n<\/tr>\n<tr>\n<td>3<\/td>\n<td>12<\/td>\n<td>3.86<\/td>\n<\/tr>\n<tr>\n<td>4<\/td>\n<td>13<\/td>\n<td>4.07<\/td>\n<\/tr>\n<tr>\n<td>5<\/td>\n<td>14<\/td>\n<td>4.57<\/td>\n<\/tr>\n<tr>\n<td>6<\/td>\n<td>15<\/td>\n<td>4.86<\/td>\n<\/tr>\n<tr>\n<td>7<\/td>\n<td>16<\/td>\n<td>5<\/td>\n<\/tr>\n<tr>\n<td>8<\/td>\n<td>17<\/td>\n<td>5.43<\/td>\n<\/tr>\n<tr>\n<td>9<\/td>\n<td>18<\/td>\n<td>5.64<\/td>\n<\/tr>\n<tr>\n<td>10<\/td>\n<td>19<\/td>\n<td>6.16<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p style=\"text-align: center;\"><strong>Note<\/strong>: <em>The\u00a0raw .csv file for this data is available for <a href=\"https:\/\/github.com\/alpharithms\/examples\/blob\/f099a98ceeaa76d9a868188d14e0495ac0bc11fe\/data\/covariance-sample-data.csv\">download on Github<\/a>.<\/em><\/p>\n<h2>Step 1: Calculate the Sample Mean<\/h2>\n<p>Now that we have our data we can begin the process of calculating the covariance. The first step is to calculate the sample mean for both variables in our data. The formula for this calculation is as such:<\/p>\n<figure id=\"attachment_1112\" aria-describedby=\"caption-attachment-1112\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance.jpg\"><img decoding=\"async\" class=\"wp-image-1112 size-large\" title=\"sample-mean-formula-covariance\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance-1024x576.jpg\" alt=\"sample mean formula covariance\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance-1024x576.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance-780x439.jpg 780w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-formula-covariance.jpg 1365w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1112\" class=\"wp-caption-text\">The formulae for calculating the sample means of our X &amp; Y variable series. (click to enlarge)<\/figcaption><\/figure>\n<p>Mathematical formulae have a tendency to come off as intimidating and formal. These represent nothing more complex than saying &#8220;add all the numbers up and divide by how many are observed.&#8221; In other words\u2014the <em>average<\/em> everyone knows and loves (arithmetic mean.) Consider the following illustration of this process:<\/p>\n<figure id=\"attachment_1113\" aria-describedby=\"caption-attachment-1113\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-1113\" title=\"sample-mean-calculation-illustration\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration-1024x576.jpg\" alt=\"sample mean calculation illustration\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration-1024x576.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration-780x439.jpg 780w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/sample-mean-calculation-illustration.jpg 1365w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1113\" class=\"wp-caption-text\">Applying the formulae before we realize the sample mean values of both variable series. (click to enlarge)<\/figcaption><\/figure>\n<h2>Step 2 (optional): Calculate Signs of Sample Mean Relationships<\/h2>\n<p>By calculating the sign (positive vs. negative) of the relationship between each of our sample variables and their respective mean we can glean some quick insight. This step, arguably optional, describes whether the observed value of one variable might increase vs. decrease in relation to the other.<\/p>\n<figure id=\"attachment_1114\" aria-describedby=\"caption-attachment-1114\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-1114\" title=\"covariance-sample-means-sign-chart\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart-1024x576.jpg\" alt=\"covariance sample means sign chart\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart-1024x576.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart-780x439.jpg 780w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-sample-means-sign-chart.jpg 1365w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1114\" class=\"wp-caption-text\">Calculation of each variable&#8217;s observed valued minus its respective sampled mean offers an illustrative representation of covariance sign. (click to enlarge)<\/figcaption><\/figure>\n<p>If there are more points in quadrants 1 and 3 the relationship between variables X and Y is a <strong>positive linear relationship<\/strong>. If there are more points in quadrants 2 and 4 the relationship between X and Y is a <strong>negative linear relationship<\/strong>. As a further illustration of the relationship between the sampled variable&#8217;s values and their respective sample means the following table details the resulting signs:<\/p>\n<figure id=\"attachment_1120\" aria-describedby=\"caption-attachment-1120\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-1120\" title=\"covariance-quadrant-sign-chart\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart-1024x576.jpg\" alt=\"covariance quadrant sign chart\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart-1024x576.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart-780x439.jpg 780w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-quadrant-sign-chart.jpg 1365w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1120\" class=\"wp-caption-text\">This chart details the signs associated with resulting quadrants based on sampled values&#8217; relationship to their respective means. (click to enlarge)<\/figcaption><\/figure>\n<p>When a scatterplot of these data are not available one can use a chart similar to this one to assess whether the relationship between X and Y is positive or negative. This assessment is done by adding the values of the last column. If the value is positive, the linear relationship between X and Y positive. If the value is negative, the linear relationship between X and Y is negative. This can also be used when the scatterplot doesn&#8217;t show a clear linear relationship (low r-value.)<\/p>\n<h2>Step 3: Calculate the Covariance<\/h2>\n<p>In the above scatter plot note that more points are observed in the 1<sup>st<\/sup> and 3<sup>rd<\/sup> quadrants. This observation indicates a positive linear relationship between the values of x and y. This is supported by summing the results of the (y<sub>i<\/sub> &#8211; \u0233)(x<sub>i<\/sub>-x\u0304) column values which results in the positive value of ~26.61 in the case of our sample dataset. The values for this calculation can be seen in the table below:<\/p>\n<table id=\"covariance-calculations\" class=\"centered-text\">\n<thead>\n<tr>\n<th>Observation<br \/>\nNumber<\/th>\n<th>Predictor<br \/>\n(X)<\/th>\n<th>x<sub>i<\/sub>-x\u0304<\/th>\n<th>Response<br \/>\n(Y)<\/th>\n<th>y<sub>i<\/sub> &#8211; \u0233<\/th>\n<th>(y<sub>i<\/sub> &#8211; \u0233)(x<sub>i<\/sub>-x\u0304)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>1<\/td>\n<td>10<\/td>\n<td>4.5<\/td>\n<td>3.15<\/td>\n<td>-1.471<\/td>\n<td>-6.6195<\/td>\n<\/tr>\n<tr>\n<td>2<\/td>\n<td>11<\/td>\n<td>5.5<\/td>\n<td>3.47<\/td>\n<td>-1.151<\/td>\n<td>-6.3305<\/td>\n<\/tr>\n<tr>\n<td>3<\/td>\n<td>12<\/td>\n<td>6.5<\/td>\n<td>3.86<\/td>\n<td>-0.761<\/td>\n<td>-4.9465<\/td>\n<\/tr>\n<tr>\n<td>4<\/td>\n<td>13<\/td>\n<td>7.5<\/td>\n<td>4.07<\/td>\n<td>-0.551<\/td>\n<td>-4.1325<\/td>\n<\/tr>\n<tr>\n<td>5<\/td>\n<td>14<\/td>\n<td>8.5<\/td>\n<td>4.57<\/td>\n<td>-0.051<\/td>\n<td>-0.4335<\/td>\n<\/tr>\n<tr>\n<td>6<\/td>\n<td>15<\/td>\n<td>9.5<\/td>\n<td>4.86<\/td>\n<td>0.239<\/td>\n<td>2.2705<\/td>\n<\/tr>\n<tr>\n<td>7<\/td>\n<td>16<\/td>\n<td>10.5<\/td>\n<td>5<\/td>\n<td>0.379<\/td>\n<td>3.9795<\/td>\n<\/tr>\n<tr>\n<td>8<\/td>\n<td>17<\/td>\n<td>11.5<\/td>\n<td>5.43<\/td>\n<td>0.809<\/td>\n<td>9.3035<\/td>\n<\/tr>\n<tr>\n<td>9<\/td>\n<td>18<\/td>\n<td>12.5<\/td>\n<td>5.64<\/td>\n<td>1.019<\/td>\n<td>12.7375<\/td>\n<\/tr>\n<tr>\n<td>10<\/td>\n<td>19<\/td>\n<td>13.5<\/td>\n<td>6.16<\/td>\n<td>1.539<\/td>\n<td>20.7765<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p style=\"text-align: center;\"><strong>Note<\/strong>: <em>The data from this table is available for <a href=\"https:\/\/github.com\/alpharithms\/examples\/blob\/0808e35cb0afd60b57ef8649fd78b37148dbdddb\/data\/covariance-calculations.csv\">download on Github<\/a><\/em><\/p>\n<p>This value still needs to be standardized before it becomes useful in predicting the relationship between x and y. This is done by dividing by the total number of samples minus one\u2014culminating in our successful execution of the covariance formula. Below is a visual depiction of which aspects of the formula for covariance these values represent.<\/p>\n<figure id=\"attachment_1117\" aria-describedby=\"caption-attachment-1117\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1117 size-large\" title=\"covariance-formula-data-calculation\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1-1024x576.jpg\" alt=\"covariance formula data calculation 1\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1-1024x576.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1-780x439.jpg 780w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-formula-data-calculation-1.jpg 1365w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1117\" class=\"wp-caption-text\">The covariance utilizes the data we&#8217;ve calculated thus far to calculate an expected value of each variable&#8217;s observed values relative to their respective sample means. (click to enlarge)<\/figcaption><\/figure>\n<p>Plugging the above data and resulting calculations into the covariance formula returns a value of ~2.96\u2014the <strong>covariance of x and y<\/strong>! We&#8217;ve approached this in such a painstakingly granular way solely for the thrills and for the purpose of education. In practice, covariance is calculated with more modern tools such as those available in Python. Now that we can appreciate all the hard work these tools do for us let&#8217;s consider how to set them up!<\/p>\n<h2>Calculating the Covariance in Python<\/h2>\n<p>Python can be used to calculate the covariance of two series in a number of ways. Below are several approaches to calculating the covariance in Python utilizing popular scientific libraries like NumPy and Pandas. In addition, we&#8217;ll show how to implement a covariance calculation as a custom function. Just as a quick refresher, this is our current data:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\"># Predictor variable series\r\nx = [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]\r\n\r\n# Response variable series\r\ny = [3.15, 3.47, 3.86, 4.07, 4.57, 4.86, 5, 5.43, 5.64, 6.16]<\/pre>\n<h3>Option 1: Covariance with Pandas<\/h3>\n<p>Pandas offers several ways to calculate covariance. We&#8217;ll start with the simplest approach utilizing the <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">Series.cov()<\/code> method:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">import pandas as pd\r\n\r\n# Use the .cov() method to calculate\r\n# the covariance of the variable series\r\ncov = pd.Series(x).cov(pd.Series(y))\r\n\r\n# Print the return value and type\r\nprint(cov, type(cov))\r\n\r\n# Note the type is a numpy class\r\n2.956111111111111 &lt;class 'numpy.float64'&gt;\r\n\r\n<\/pre>\n<p>The <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">Series.cov()<\/code> method is a straightforward approach when one is starting with raw data such as <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">lists<\/code> or <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">dicts<\/code>. Check out the <a href=\"https:\/\/pandas.pydata.org\/docs\/reference\/api\/pandas.Series.cov.html\">official documentation<\/a> for more information on how to specify minimum observations and degrees of freedom. Note that this method results in a type <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">numpy.float64<\/code>. This is because Pandas <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">DataFrame<\/code>, <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">Series<\/code>, and <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">Index<\/code> objects <a href=\"https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/arrays.html\">rely <em>heavily<\/em> on NumPy arrays<\/a> under the hood.<\/p>\n<p>Pandas also has a <code class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">Dataframe.cov()<\/code> method that calculates covariance which returns a <a href=\"https:\/\/www.itl.nist.gov\/div898\/handbook\/pmc\/section5\/pmc541.htm\">covariance matrix<\/a>. This type of approach is more applicable to multivariate data so we&#8217;ll skip over that for now. The NumPy method for calculating covariance will offer a sneak-peek into that data structure.<\/p>\n<h3>Option 2: Covariance with NumPy<\/h3>\n<p>The NumPy.cov() function calculates a covariance matrix for a number of variable series. It is a bit like using a sledgehammer to drive a penny nail in the case of calculating the covariance for only two variables. Nonetheless, it&#8217;s useful to be aware of:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">import numpy as np\r\n\r\n# Generate the covariance matrix\r\ncov_matrix = np.cov(x, y)\r\n\r\n# Print the result\r\nprint(cov_matrix)\r\n\r\n# Results\r\n[[9.16666667 2.95611111]\r\n [2.95611111 0.95956556]]\r\n\r\n# Get the value we're interested in \r\ncov = cov_matrix[0][1]\r\n\r\n# Print that value + type\r\nprint(cov, type(cov))\r\n\r\n# Results\r\n2.956111111111111 &lt;class 'numpy.float64'&gt;<\/pre>\n<p>This method of calculating the covariance produces a matrix of values (remember we skipped that with Pandas.) We aren&#8217;t going to discuss the specifics of covariance matrices here but will note they are useful in identifying potential correlations between large numbers of variables (multivariate data.) While overkill for our example, we can still extract the covariance between x and y by indexing into the matrix (2D array).<\/p>\n<h3>Option 3: Custom Covariance Function<\/h3>\n<p>Python contains all the tools one needs to implement the covariance calculation using only built-in functions. Below is a custom implementation that first calculates the sample means, then the product of the difference of variables and their respective sample means, and returns that valued divided by the total sample size minus one:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\"># ensure series contain same number of elements\r\nif len(x) != len(y):\r\n    raise Exception(\"Series must be of equal length!\")\r\n\r\n# Calculate the sample means once\r\nx_mean = sum(x) \/ len(x)\r\ny_mean = sum(y) \/ len(y)\r\n\r\n# Calculate the sum of the product between\r\n# the differences of observed values and their\r\n# respective sample means.\r\ncov = 0.0\r\nfor xi, yi in zip(x, y):\r\n    cov += (xi - x_mean) * (yi - y_mean)\r\n\r\n# Divide the covariance by the \r\n# number of samples minus 1\r\ncov = cov \/ (len(x) - 1)\r\n\r\n# Print the result\r\nprint(cov, type(cov))\r\n\r\n# Result\r\n2.9561111111111114 &lt;class 'float'&gt;\r\n<\/pre>\n<p style=\"text-align: center;\"><em><strong>Note<\/strong>: This code is <a href=\"https:\/\/github.com\/alpharithms\/statistics\/blob\/83eff745faad981571a139ec83ff227b08ceaeab\/covariance.py\">available as a function via Github<\/a><\/em><\/p>\n<h2>Examples<\/h2>\n<p>Its important to always remember that covariance only measures the direction and not magnitude. This limits the utility of the measure but does not negate it entirely. Covariance can help with early-stage feature selection by identifying relationships between variables in a computationally cheaper manner than full-scale correlation analysis. Below is an example of a covariance matrix to visualize the covariance between different features in a multivariate model (Zhou, 2026):<\/p>\n<div class=\"mceTemp\"><\/div>\n<figure id=\"attachment_1124\" aria-describedby=\"caption-attachment-1124\" style=\"width: 1024px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-1124\" title=\"covariance-matrix-heatmaps\" src=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps-1024x576.jpg\" alt=\"covariance matrix heatmaps\" width=\"1024\" height=\"576\" srcset=\"https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps-1024x576.jpg 1024w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps-300x169.jpg 300w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps-768x432.jpg 768w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps-450x253.jpg 450w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps-780x439.jpg 780w, https:\/\/www.alpharithms.com\/wp-content\/uploads\/1089\/covariance-matrix-heatmaps.jpg 1365w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption id=\"caption-attachment-1124\" class=\"wp-caption-text\">Heatmap covariance matrix illustrating the relationship between different features. (click to enlarge)<\/figcaption><\/figure>\n<h2>Applications<\/h2>\n<p>Covariance is frequently used to analyze stock prices relative to one another or between price and common indicators such as earnings reports (Ghorbani, 2020). For example, the market price of a stock tends to increase with positive earnings reports and negatively on negative earnings reports.<\/p>\n<p>Covariance is utilized in the application of <a href=\"https:\/\/www.investopedia.com\/terms\/m\/modernportfoliotheory.asp\">Modern Portfolio Theory (MPT)<\/a> where it has been used to balance assets to achieve the desired level of risk for decades (Mercurio, 2020). For example, assets exhibiting negative covariance are inherently less-risky since a loss of one is likely to be offset by gains in the other. Covariance, among other statistical methods, allows MPT to assess the performance of an asset relative to its impact on the portfolio as a whole.<\/p>\n<p>Covariance is also used in the <a href=\"https:\/\/www.investopedia.com\/terms\/c\/capm.asp\">Capital Asset Pricing Model (CAPM)<\/a> much in the way it is utilized by MPT\u2014to reduce risk. CAPM applies an approach known as Time-varying covariances that measure the performance of an asset relative to the rest of the portfolio over a period of time. This calculation is often made within a larger analysis such as multivariate regression (Bollerslev, 2021, <span class=\"authors\">Acosta-Gonzalez, 2008<\/span>).<\/p>\n<h2>Review<\/h2>\n<p>Covariance is a fundamental tool in describing the nature of relationships in mathematical terms. We&#8217;ve seen how it can be manually calculated as well as derived from powerful modern tools like Python\u2014either natively or via powerful scientific packages like NumPy or Pandas (or both!)<\/p>\n<p>Covariance is a useful tool whether you are selecting portfolio assets to meet your risk level or selecting features in your regression model. Covariance is but one step in a larger discipline of correlation analysis and plays an integral role in the development of many highly utilized machine learning algorithms and frameworks within the <a href=\"https:\/\/www.alpharithms.com\/most-popular-programming-languages-184015\/\">most popular programming languages today<\/a>.<\/p>\n<h2>References<\/h2>\n<ol>\n<li>Gourlay, Neil.<i> \u201c<\/i>Covariance Analysis And Its Applications In Psychological Research.\u201d <i>British Journal of Statistical Psychology<\/i>, vol. 6, no. 1, 1953, pp. 25\u201334. <i>Crossref<\/i>, doi:10.1111\/j.2044-8317.1953.tb00128.x.<\/li>\n<li>Bartz, Daniel et al. \u201cDirectional variance adjustment: bias reduction in covariance matrices based on factor analysis with an application to portfolio optimization.\u201d\u00a0<i>PloS one<\/i>\u00a0vol. 8,7 e67503. 3 Jul. 2013, doi:10.1371\/journal.pone.0067503<\/li>\n<li>Jang, Eunice Eunhee, and Louis Roussos. \u201cAn Investigation into the Dimensionality of TOEFL Using Conditional Covariance-Based Nonparametric Approach.\u201d <i>Journal of Educational Measurement<\/i>, vol. 44, no. 1, 2007, pp. 1\u201321. <i>Crossref<\/i>, doi:10.1111\/j.1745-3984.2007.00024.x.<\/li>\n<li>Schlotz, Wolff et al. \u201cCovariance between psychological and endocrine responses to pharmacological challenge and psychosocial stress: a question of timing.\u201d\u00a0<i>Psychosomatic medicine<\/i>\u00a0vol. 70,7 (2008): 787-96. doi:10.1097\/PSY.0b013e3181810658<\/li>\n<li>Ghorbani, Mahsa, and Edwin K P Chong. \u201cStock price prediction using principal components.\u201d\u00a0<i>PloS one<\/i>\u00a0vol. 15,3 e0230124. 20 Mar. 2020, doi:10.1371\/journal.pone.0230124<\/li>\n<li>Bollerslev, Tim, et al. \u201cA Capital Asset Pricing Model with Time-Varying Covariances.\u201d\u00a0<i>Journal of Political Economy<\/i>, vol. 96, no. 1, 1988, pp. 116\u2013131.\u00a0<i>JSTOR<\/i>, www.jstor.org\/stable\/1830713. Accessed 14 July 2021.<\/li>\n<li><span class=\"authors\">Acosta-Gonzalez, Eduardo, et. al.<\/span> &#8220;<span class=\"art_title\">Estimating time-varying variances and covariances via nearest neighbor multivariate predictions: applications to the NYSE and the Madrid Stock Exchange Index.&#8221;<\/span>\u00a0<span class=\"serial_title\"><em>Applied Economics<\/em>,<\/span>\u00a0<span class=\"volume_issue\">41:26,<\/span>\u00a0<span class=\"page_range\">3437-3445,<\/span> 2009. <span class=\"doi_link\">doi:10.1080\/00036840701439371<\/span><\/li>\n<li>Zhou, Yi-Hui. \u201cA robust covariance testing approach for high-throughput data.\u201d\u00a0<em>arXiv: Methodology<\/em>\u00a0(2016): n. pag.<\/li>\n<li>Mercurio, Peter Joseph et al. \u201cAn Entropy-Based Approach to Portfolio Optimization.\u201d\u00a0<i>Entropy (Basel, Switzerland)<\/i>\u00a0vol. 22,3 332. 14 Mar. 2020, doi:10.3390\/e22030332<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Covariance is a statistical method for describing how two variables change in relationship to one another. Its measure is used to describe directional correlation such that the change in observed value for one variable is reflected by a change in another. Covariance is integral to modern statistics but often only as accompanying support within more [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":1130,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[655,195,260],"tags":[439,433,198,210,440,420],"acf":[],"_links":{"self":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/1089"}],"collection":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/comments?post=1089"}],"version-history":[{"count":10,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/1089\/revisions"}],"predecessor-version":[{"id":1173,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/1089\/revisions\/1173"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media\/1130"}],"wp:attachment":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media?parent=1089"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/categories?post=1089"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/tags?post=1089"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}