Covariance and correlation: why a mix is safer than its parts

Correlation shows up as soon as you look at more than one stock, and it’s at the heart of portfolio construction. The mean and standard deviation post measured the risk of one fund. This one answers the question every portfolio manager asks next: what happens to risk when you hold two things together?

Covariance: do two things move together?

Plain meaning: on days when A is above its average, is B also above its average (positive), below it (negative), or neither (around zero)?

how two things move together: ρ and covrisk: σ, s and the covariance matrix Σ

$$\key{\text{cov}}(A, B) = \frac{1}{n-1} \sum_{t=1}^{n} (a_t - \bar{a})(b_t - \bar{b})$$

Look at the product $(a_t - \bar{a})(b_t - \bar{b})$. If both are above average, or both below, the product is positive. If one is up while the other is down, it’s negative. Averaging the products tells you which happens more. (It’s the same $n - 1$ as in the sample standard deviation. In fact, the covariance of a thing with itself is its variance.)

Probability books write the same idea for random variables. Bishop and Bishop’s Deep Learning: Foundations and Concepts (equation 2.48 is the vector version) gives, for two quantities $x$ and $y$:

$$\key{\text{cov}}[x, y] = E[xy] - E[x]\,E[y]$$

“the average of the product, minus the product of the averages”. The sum above is how you estimate it from data.

Worked example: SPY and gold

The first five trading days of 2020, SPY (the S&P 500 fund) and GLD (gold), daily returns in percent rounded to one decimal, the same data as the mean and standard deviation post (Table 1):

$$\key{\text{cov}}(A, B) = \frac{-1.71}{5 - 1} = -0.43$$

Negative: that week, when stocks had a good day, gold tended to have a bad one, and the other way round.

But what does −0.43 mean? Is that a lot? Covariance is hard to read, because its size depends on the units. Returns in percent give a number 10,000 times bigger than returns as decimals. That’s why we usually use correlation.

Correlation: covariance on a fixed scale

Divide the covariance by both standard deviations:

$$\key{\rho_{A,B}} = \frac{\key{\text{cov}}(A, B)}{\risk{s_A} \, \risk{s_B}}$$

The result, $\key{\rho}$ (“rho”), always lies between −1 and +1 (Table 2):

For that week, $\risk{s_A} = 0.628$ and $\risk{s_B} = 0.937$:

$$\key{\rho} = \frac{-0.43}{0.628 \times 0.937} = -0.73$$

Strongly opposite, in that one week. Five days is far too few to trust; over the whole of 2020 to 2025 the SPY–GLD correlation is only $\key{0.13}$, and it moves around. Table 3 is the full matrix for 2020 to 2025, each daily return against each other:

The stocks move together (0.2 to 0.8; SPY correlates with every one of them because it contains them). Gold moves on its own (near 0 with everything). That is the reason gold is in so many portfolios. Not its return, but what it does in the next formula.

Two warnings:

• Correlation only sees straight-line links. Two things can be strongly related in a curved way and still show a correlation near 0.
• Correlation is not causation. Two series can move together because a third thing drives both.

Portfolio variance: where correlation pays

Put a share $w_1$ of your money in asset 1 and $w_2 = 1 - w_1$ in asset 2. The portfolio’s variance is (Yan, Python for Finance, chapter 9, equation 3; Table 4 names the symbols):

$$\risk{\sigma_p^2} = w_1^2 \risk{\sigma_1^2} + w_2^2 \risk{\sigma_2^2} + 2\, w_1 w_2\, \key{\rho}\, \risk{\sigma_1} \risk{\sigma_2}$$

The first two terms are each asset’s own risk, scaled by how much you hold. The third term is where correlation acts. With $\key{\rho}$ below 1, part of the two risks cancels out. With $\key{\rho}$ negative, the third term is negative and cancels even more.

Worked example: 60% SPY, 40% gold

From 2020 to 2025: SPY’s yearly standard deviation is 20.9%, GLD’s 16.3%, and their correlation 0.13. Held 60/40:

$$\begin{aligned} \risk{\sigma_p^2} &= 0.6^2 \times 0.209^2 \\ &\quad + 0.4^2 \times 0.163^2 \\ &\quad + 2 \times 0.6 \times 0.4 \times 0.13 \times 0.209 \times 0.163 \end{aligned}$$ $$\risk{\sigma_p^2} = 0.015725 + 0.004251 + 0.002126$$ $$\risk{\sigma_p^2} = 0.022102$$ $$\risk{\sigma_p} = \sqrt{0.022102}$$ $$\risk{\sigma_p} = 14.87\%$$

If you just averaged the two risks ($0.6 \times 20.9\% + 0.4 \times 16.3\%$), you’d get $\risk{19.1\%}$. The formula says $\risk{14.9\%}$, and the actual standard deviation of that 60/40 mix, rebalanced daily over 2020 to 2025, was 14.87%. That gap is the diversification benefit. The plain weighted average is only right when the correlation is exactly +1.

Each curve in Figure 1 is one correlation. At $\key{+1}$ (grey, straight), mixing doesn’t reduce risk at all. The lower the correlation, the more the curve sags below the straight line; the solid red curve is SPY and GLD’s actual 0.13. At $\key{-1}$ (dashed red) it touches zero: a perfect hedge exists.

There’s a mix with the lowest possible risk. For two assets:

$$w_1^{*} = \frac{\risk{\sigma_2^2} - \key{\rho} \risk{\sigma_1} \risk{\sigma_2}}{\risk{\sigma_1^2} + \risk{\sigma_2^2} - 2 \key{\rho} \risk{\sigma_1} \risk{\sigma_2}}$$

Here that’s $\key{36.1\%}$ in SPY, with a standard deviation of $\risk{13.6\%}$. That’s lower than holding only gold (16.3%), even though you’ve added the riskier asset. Adding something risky made the whole thing safer, because the two barely move together.

Correlation is not a constant

The correlations above are one number per pair for 2020 to 2025. Measured over a sliding 60-day window they wander, and they wander in a particular direction when it matters (Table 5):

In a crash, correlations between stocks rise towards one. SPY and NVDA hit 0.90 in the window that ends in April 2020 (Figure 2). GS and XOM, a bank and an oil company with little in common, stood at 0.84 at the end of March 2020, a hair below their 0.85 peak the day before, in the window that covers the crash. The year-by-year version, for an equal-weight mix of SPY, NVDA, GS and XOM, is Table 6:

Diversification did least in the year it was needed most, because the thing that was driving every stock down was the same thing. That is the difference between specific risk (one company’s news, which averages away across holdings) and systematic risk (the whole economy, which does not). With many assets, the portfolio variance tends to the average covariance between them: own variances wash out, shared movement stays. The floor on a portfolio’s volatility is roughly $\sqrt{\key{\rho}}\,\risk{\sigma}$ for assets of equal volatility, and the bagging post meets exactly the same floor when it averages correlated models.

Many assets: the matrix form

With more than two assets, Hilpisch’s Python for Finance (chapter 13, equation 13-2) writes the variance as a sum over every pair of assets of $w_i w_j \sigma_{ij}$, where the covariance $\sigma_{ij}$ is $\rho_{ij}\sigma_i\sigma_j$:

$$\risk{\sigma_p^2} = \sum_i \sum_j w_i\, w_j\, \risk{\sigma_i}\, \risk{\sigma_j}\, \key{\rho_{ij}}$$

where $\key{\rho_{ii}} = 1$, so each asset’s own variance appears when $i = j$. The compact way to write it: put the weights in a vector $w$ and all variances and covariances in a covariance matrix $\risk{\Sigma}$:

$$\risk{\sigma_p^2} = w^\top \risk{\Sigma}\, w$$ $$\risk{\Sigma} = \begin{pmatrix} \risk{\sigma_1^2} & \key{\rho}\,\risk{\sigma_1}\risk{\sigma_2} \\ \key{\rho}\,\risk{\sigma_1}\risk{\sigma_2} & \risk{\sigma_2^2} \end{pmatrix}$$

The diagonal holds each asset’s variance, and the off-diagonal holds the covariances. For the 60/40 example, $w^\top \risk{\Sigma} w = 0.022102$, the same as the long formula.

One more useful result. Hold $n$ assets in equal amounts, each with standard deviation $\risk{\sigma}$ and the same correlation $\key{\rho}$ between every pair:

$$\risk{\sigma_p^2} = \frac{\risk{\sigma^2}}{n} + \left(1 - \frac{1}{n}\right) \key{\rho}\, \risk{\sigma^2}$$

As $n$ grows, the first term disappears. What’s left is $\key{\rho}\, \risk{\sigma^2}$, so you can never get the risk below $\sqrt{\key{\rho}}\, \risk{\sigma}$ (Table 7):

With zero correlation, diversification removes almost all the risk. With 0.3, going from 10 assets to 100 barely helps: the floor is $\sqrt{0.3} \times 20\% = 11.0\%$. That floor is market risk, the part you can’t diversify away.

One catch. Correlations are not fixed. In a crisis, many assets fall together and correlations jump up, exactly when you need diversification most.

The code

The correlation matrix, the 60/40 risk two ways, and the lowest-risk mix. It reads the blog’s price file from the examples repo, so it runs as pasted (pip install numpy pandas). returns holds the daily returns, one column per ticker; Sigma the yearly covariance matrix; w the weights.

import numpy as np
import pandas as pd

URL = "https://raw.githubusercontent.com/ankurtrapasiya/examples/main/data/prices.csv"
prices = pd.read_csv(URL, index_col="date", parse_dates=True)
returns = (prices / prices.shift(1) - 1).dropna()  # daily returns, one column per ticker

print(returns.corr().round(2))                   # the 5 x 5 correlation matrix
rho = returns["SPY"].corr(returns["GLD"])        # 0.13
Sigma = returns.cov() * 252                      # yearly covariance matrix: variance grows with days

# 60/40 SPY/GLD risk, two ways
s1 = returns["SPY"].std() * 252 ** 0.5           # 0.209: SPY's yearly standard deviation
s2 = returns["GLD"].std() * 252 ** 0.5           # 0.163: GLD's
w1, w2 = 0.6, 0.4
var_p = w1**2 * s1**2 + w2**2 * s2**2 + 2 * w1 * w2 * rho * s1 * s2      # the long formula
w = pd.Series({"SPY": 0.6, "GLD": 0.4})
print(round(np.sqrt(var_p), 4), round(np.sqrt(w @ Sigma.loc[w.index, w.index] @ w), 4))   # 0.1487 both

# The mix with the lowest risk
w_star = (s2**2 - rho * s1 * s2) / (s1**2 + s2**2 - 2 * rho * s1 * s2)
sd_star = np.sqrt(w_star**2 * s1**2 + (1 - w_star)**2 * s2**2 + 2 * w_star * (1 - w_star) * rho * s1 * s2)
print(f"{w_star:.1%} in SPY, risk {sd_star:.1%}")   # 35.9%, 13.6%

It prints the correlation matrix above, then 0.1487 twice (the long formula and the matrix form agree), then about 35.9% in SPY at 13.6% risk. The hand calculation above used inputs rounded to 0.209, 0.163 and 0.13, which gives 14.87% and 36.1%; the code uses the unrounded ones. @ is matrix multiplication, so w @ Sigma @ w is $w^\top \Sigma w$.

Summary

Table 8 collects the formulas.

References

• Christopher M. Bishop and Hugh Bishop, Deep Learning: Foundations and Concepts (Springer, 2024), chapter 2, section 2.2.2: covariance (equation 2.48)
• Yuxing Yan, Python for Finance (2nd ed., Packt, 2017), chapter 9: the two-asset portfolio variance (equation 3), and sample covariance and correlation (equations 4 and 5)
• Yves Hilpisch, Python for Finance: Mastering Data-Driven Finance (2nd ed., O’Reilly, 2018), chapter 13: the general portfolio variance (equation 13-2)
• Philip J. Romero and Tucker Balch, What Hedge Funds Really Do: An Introduction to Portfolio Management (Business Expert Press, 2014), chapter 10: why low correlation reduces portfolio risk
• Harry Markowitz, “Portfolio Selection”, The Journal of Finance 7(1), 77–91, 1952
• NumPy cov and corrcoef