Covariance and correlation: why a mix is safer than its parts

Correlation shows up as soon as you look at more than one stock, and it’s at the heart of portfolio construction. The mean and standard deviation post measured the risk of one fund. This one answers the question every portfolio manager asks next: what happens to risk when you hold two things together?

Covariance: do two things move together?

Plain meaning: on days when A is above its average, is B also above its average (positive), below it (negative), or neither (around zero)?

how two things move together: ρ and covrisk: σ, s and the covariance matrix Σ

$$\key{\text{cov}}(A, B) = \frac{1}{n-1} \sum_{t=1}^{n} (a_t - \bar{a})(b_t - \bar{b})$$

Look at the product $(a_t - \bar{a})(b_t - \bar{b})$. If both are above average, or both below, the product is positive. If one is up while the other is down, it’s negative. Averaging the products tells you which happens more. (It’s the same $n - 1$ as in the sample standard deviation. In fact, the covariance of a thing with itself is its variance.)

Probability books write the same idea for random variables. Bishop and Bishop’s Deep Learning: Foundations and Concepts (equation 2.48 is the vector version) gives, for two quantities $x$ and $y$:

$$\key{\text{cov}}[x, y] = E[xy] - E[x]\,E[y]$$

“the average of the product, minus the product of the averages”. The sum above is how you estimate it from data.

Worked example: SPY and gold

The first five trading days of 2019, SPY (the S&P 500 fund) and GLD (gold), daily returns in percent rounded to one decimal, the same data as the mean and standard deviation post:

$$\key{\text{cov}}(A, B) = \frac{-4.96}{5 - 1} = -1.24$$

Negative: that week, when stocks had a good day, gold tended to have a bad one, and the other way round.

But what does −1.24 mean? Is that a lot? Covariance is hard to read, because its size depends on the units. Returns in percent give a number 10,000 times bigger than returns as decimals. That’s why we usually use correlation.

Correlation: covariance on a fixed scale

Divide the covariance by both standard deviations:

$$\key{\rho_{A,B}} = \frac{\key{\text{cov}}(A, B)}{\risk{s_A} \, \risk{s_B}}$$

The result, $\key{\rho}$ (“rho”), always lies between −1 and +1:

For that week, $\risk{s_A} = 2.027$ and $\risk{s_B} = 0.688$:

$$\key{\rho} = \frac{-1.24}{2.027 \times 0.688} = -0.89$$

Almost perfectly opposite, in that one week. Five days is far too few to trust; over the whole five years the SPY–GLD correlation is only $\key{0.11}$, and it moves around. Here is the full matrix for 2019 to 2023, each daily return against each other:

The stocks move together (0.3 to 0.8, with SPY highest because it contains the others). Gold moves on its own (near 0 with everything). That is the reason gold is in so many portfolios. Not its return, but what it does in the next formula.

Two warnings:

• Correlation only sees straight-line links. Two things can be strongly related in a curved way and still show a correlation near 0.
• Correlation is not causation. Two series can move together because a third thing drives both.

Portfolio variance: where correlation pays

Put a share $w_1$ of your money in asset 1 and $w_2 = 1 - w_1$ in asset 2. The portfolio’s variance is (Romero and Balch, What Hedge Funds Really Do, chapter 10):

$$\risk{\sigma_p^2} = w_1^2 \risk{\sigma_1^2} + w_2^2 \risk{\sigma_2^2} + 2\, w_1 w_2\, \key{\rho}\, \risk{\sigma_1} \risk{\sigma_2}$$

The first two terms are each asset’s own risk, scaled by how much you hold. The third term is where correlation acts. With $\key{\rho}$ below 1, part of the two risks cancels out. With $\key{\rho}$ negative, the third term is negative and cancels even more.

Worked example: 60% SPY, 40% gold

From the five years of data: SPY’s yearly standard deviation is 21.0%, GLD’s 14.9%, and their correlation 0.11. Held 60/40:

$$\begin{aligned} \risk{\sigma_p^2} &= 0.6^2 \times 0.210^2 \\ &\quad + 0.4^2 \times 0.149^2 \\ &\quad + 2 \times 0.6 \times 0.4 \times 0.11 \times 0.210 \times 0.149 \end{aligned}$$ $$\begin{aligned} \risk{\sigma_p^2} &= 0.015876 + 0.003552 + 0.001652 \\ &= 0.021080 \end{aligned}$$ $$\begin{aligned} \risk{\sigma_p} &= \sqrt{0.021080} \\[10pt] &= 14.52\% \end{aligned}$$

If you just averaged the two risks ($0.6 \times 21.0\% + 0.4 \times 14.9\%$), you’d get $\risk{18.6\%}$. The formula says $\risk{14.5\%}$, and the actual standard deviation of that 60/40 mix, rebalanced daily over the five years, was 14.51%. That gap is the diversification benefit. The plain weighted average is only right when the correlation is exactly +1.

Each curve is one correlation. At $\key{+1}$ (grey, straight), mixing doesn’t reduce risk at all. The lower the correlation, the more the curve sags below the straight line; the solid red curve is SPY and GLD’s actual 0.11. At $\key{-1}$ (dashed red) it touches zero: a perfect hedge exists.

There’s a mix with the lowest possible risk. For two assets:

$$w_1^{*} = \frac{\risk{\sigma_2^2} - \key{\rho} \risk{\sigma_1} \risk{\sigma_2}}{\risk{\sigma_1^2} + \risk{\sigma_2^2} - 2 \key{\rho} \risk{\sigma_1} \risk{\sigma_2}}$$

Here that’s $\key{31.6\%}$ in SPY, with a standard deviation of $\risk{12.8\%}$. That’s lower than holding only gold (14.9%), even though you’ve added the riskier asset. Adding something risky made the whole thing safer, because the two barely move together.

Correlation is not a constant

The correlations above are one number per pair for five years. Measured over a sliding 60-day window they wander, and they wander in a particular direction when it matters:

In a crash, correlations rise towards one. JPM and XOM, a bank and an oil company with little in common, moved almost in lockstep in March 2020. The year-by-year version, for an equal-weight mix of SPY, AAPL, JPM and XOM:

Diversification did least in the year it was needed most, because the thing that was driving every stock down was the same thing. That is the difference between specific risk (one company’s news, which averages away across holdings) and systematic risk (the whole economy, which does not). With many assets, the portfolio variance tends to the average covariance between them: own variances wash out, shared movement stays. The floor on a portfolio’s volatility is roughly $\sqrt{\key{\rho}}\,\risk{\sigma}$ for assets of equal volatility, and the bagging post meets exactly the same floor when it averages correlated models.

Many assets: the matrix form

With more than two assets, the same book writes the variance as a sum over every pair of assets:

$$\risk{\sigma_p^2} = \sum_i \sum_j w_i\, w_j\, \risk{\sigma_i}\, \risk{\sigma_j}\, \key{\rho_{ij}}$$

where $\key{\rho_{ii}} = 1$, so each asset’s own variance appears when $i = j$. The compact way to write it: put the weights in a vector $w$ and all variances and covariances in a covariance matrix $\risk{\Sigma}$:

$$\risk{\sigma_p^2} = w^\top \risk{\Sigma}\, w$$ $$\risk{\Sigma} = \begin{pmatrix} \risk{\sigma_1^2} & \key{\rho}\,\risk{\sigma_1}\risk{\sigma_2} \\ \key{\rho}\,\risk{\sigma_1}\risk{\sigma_2} & \risk{\sigma_2^2} \end{pmatrix}$$

The diagonal holds each asset’s variance, and the off-diagonal holds the covariances. For the 60/40 example, $w^\top \risk{\Sigma} w = 0.021080$, the same as the long formula.

One more useful result. Hold $n$ assets in equal amounts, each with standard deviation $\risk{\sigma}$ and the same correlation $\key{\rho}$ between every pair:

$$\risk{\sigma_p^2} = \frac{\risk{\sigma^2}}{n} + \left(1 - \frac{1}{n}\right) \key{\rho}\, \risk{\sigma^2}$$

As $n$ grows, the first term disappears. What’s left is $\key{\rho}\, \risk{\sigma^2}$, so you can never get the risk below $\sqrt{\key{\rho}}\, \risk{\sigma}$:

With zero correlation, diversification removes almost all the risk. With 0.3, going from 10 assets to 100 barely helps: the floor is $\sqrt{0.3} \times 20\% = 11.0\%$. That floor is market risk, the part you can’t diversify away.

One catch. Correlations are not fixed. In a crisis, many assets fall together and correlations jump up, exactly when you need diversification most.

The code

The correlation matrix, the 60/40 risk two ways, and the lowest-risk mix. It reads the blog’s price file from the examples repo, so it runs as pasted (pip install numpy pandas). returns holds the daily returns, one column per ticker; Sigma the yearly covariance matrix; w the weights.

import numpy as np
import pandas as pd

URL = "https://raw.githubusercontent.com/ankurtrapasiya/examples/main/data/prices.csv"
prices = pd.read_csv(URL, index_col="date", parse_dates=True)
returns = (prices / prices.shift(1) - 1).dropna()  # daily returns, one column per ticker

print(returns.corr().round(2))                   # the 5 x 5 correlation matrix
rho = returns["SPY"].corr(returns["GLD"])        # 0.11
Sigma = returns.cov() * 252                      # yearly covariance matrix: variance grows with days

# 60/40 SPY/GLD risk, two ways
s1 = returns["SPY"].std() * 252 ** 0.5           # 0.210: SPY's yearly standard deviation
s2 = returns["GLD"].std() * 252 ** 0.5           # 0.149: GLD's
w1, w2 = 0.6, 0.4
var_p = w1**2 * s1**2 + w2**2 * s2**2 + 2 * w1 * w2 * rho * s1 * s2      # the long formula
w = pd.Series({"SPY": 0.6, "GLD": 0.4})
print(round(np.sqrt(var_p), 4), round(np.sqrt(w @ Sigma.loc[w.index, w.index] @ w), 4))   # 0.1452 both

# The mix with the lowest risk
w_star = (s2**2 - rho * s1 * s2) / (s1**2 + s2**2 - 2 * rho * s1 * s2)
sd_star = np.sqrt(w_star**2 * s1**2 + (1 - w_star)**2 * s2**2 + 2 * w_star * (1 - w_star) * rho * s1 * s2)
print(f"{w_star:.1%} in SPY, risk {sd_star:.1%}")   # 31.6%, 12.8%

It prints the correlation matrix above, then 0.1451 twice (the long formula and the matrix form agree), then about 31.7% in SPY at 12.8% risk. The hand calculation above used inputs rounded to 0.210, 0.149 and 0.11, which gives 14.52% and 31.6%; the code uses the unrounded ones. @ is matrix multiplication, so w @ Sigma @ w is $w^\top \Sigma w$.

Summary

References

• Christopher M. Bishop and Hugh Bishop, Deep Learning: Foundations and Concepts (Springer, 2024), chapter 2, section 2.2.2: covariance (equation 2.48)
• Philip J. Romero and Tucker Balch, What Hedge Funds Really Do: An Introduction to Portfolio Management (Business Expert Press, 2014), chapter 10: portfolio variance, two-asset and general forms
• Harry Markowitz, “Portfolio Selection”, The Journal of Finance 7(1), 77–91, 1952
• NumPy cov and corrcoef