Mean and standard deviation: return and risk in two numbers

Any analysis of returns starts with these two numbers. Almost every measure of a fund or strategy, including the Sharpe ratio, is built from them. The expected value post ended with “two choices can have the same average and very different risk”. This post measures that difference.

First: daily returns

In finance we rarely work with prices directly. We work with returns: the percentage change from one day to the next. This is the definition in Romero and Balch’s What Hedge Funds Really Do (chapter 2), with the symbols in Table 1:

returns, the datathe meanthe spread: standard deviation, variance

$$\good{r_t} = \frac{p_t}{p_{t-1}} - 1$$

On the first trading day of 2020, SPY (the S&P 500 fund) closed at $295.39, and the next day at $293.15: $r = 293.15/295.39 - 1 = -0.0076 = \good{-0.8\%}$. Returns let you compare a $10 stock with a $1,000 stock on the same scale. The examples below use daily closes for five tickers: SPY, NVDA, GS, XOM and GLD (gold), 2020 to 2025, adjusted for dividends and splits.

Daily returns compound: to get the return over many days, multiply one plus each day’s return together and subtract one (same chapter):

$$\text{cumulative return} = \prod_{t=1}^{n} (1 + \good{r_t}) - 1$$

Read $\prod$ as “multiply together”, the way $\sum$ means “add together”. That’s why the two funds further down end the year at different values despite the same average day, and why $100 in NVDA became $3,391 over 2020 to 2025 while the same $100 in XOM became $211.

The mean: where the middle is

Plain meaning: add everything up, divide by how many there are.

$$\key{\bar{r}} = \frac{1}{n} \sum_{t=1}^{n} \good{r_t}$$

SPY’s first five daily returns of 2020, rounded to one decimal: −0.8%, 0.4%, −0.3%, 0.5%, 0.7%.

$$\key{\bar{r}} = \frac{-0.8 + 0.4 - 0.3 + 0.5 + 0.7}{5} = \frac{0.5}{5} = 0.10\%$$

On an average day that week, SPY made $\key{0.10\%}$. (A good week: the average daily return over 2020 to 2025 is 0.065%.)

The standard deviation: how far things stray

Plain meaning: how far, typically, a day’s return lands from the average.

The recipe:

1. For each day, find how far it is from the mean (the deviation).
2. Square each deviation, so negatives don’t cancel positives.
3. Average the squares. This is the variance.
4. Take the square root, to get back to the original units (percent). This is the standard deviation.

$$\risk{s} = \sqrt{\frac{1}{n-1} \sum_{t=1}^{n} \left( \good{r_t} - \key{\bar{r}} \right)^2}$$

Worked out for the five days in Table 2 (mean 0.10%):

$$\risk{s^2} = \frac{1.58}{5 - 1}$$ $$\risk{s^2} = 0.395$$ $$\risk{s} = \sqrt{0.395}$$ $$\risk{s} = 0.63\%$$

A typical day that week landed about 0.6 percentage points away from the 0.10% average. Over the whole of 2020 to 2025 SPY’s daily standard deviation is 1.32%; that first week of January 2020 was half as jumpy as normal, two months before the Covid crash tripled it.

Why divide by $n - 1$, not $n$?

This question comes up every time. When you compute the deviations, you measure them from the sample’s own mean, which is pulled towards the data you happen to have. So the deviations come out a little too small on average. Bishop and Bishop’s Deep Learning: Foundations and Concepts shows exactly how much (equation 2.60): the divide-by-$n$ variance comes out, on average, at

$$E\left[\risk{\sigma^2_{\text{ML}}}\right] = \frac{n-1}{n}\,\risk{\sigma^2}$$

of the true variance $\risk{\sigma^2}$. Multiplying by $n/(n-1)$ undoes that, which is the same as dividing by $n - 1$ instead of $n$ (their equation 2.63). It’s called Bessel’s correction, and the result is the sample standard deviation.

Dividing by $n$ gives the population standard deviation, which is right only when you have every possible value, not a sample. With years of daily data the two versions are practically equal. For the five days above: 0.63% (sample) versus 0.56% (population). With a year of daily data the difference is tiny, but with five days it’s large.

A practical trap: NumPy’s np.std divides by $n$ by default. Pandas’ .std() divides by $n - 1$. Use np.std(x, ddof=1) to get the sample version in NumPy.

2020 to 2025, five tickers

Table 3 has the same two numbers for each ticker over 2020 to 2025, daily and then annualised (the next section explains the 252 and the square root):

Two things to notice. Gold is the calm one: the smallest spread, and a better average day than SPY. And XOM earned a better average day than SPY, 0.073% against 0.065%, yet ended at $211 against SPY’s $228; XOM’s swings, about one and a half times SPY’s, cost it more than that edge was worth, which is the next section’s point.

Same mean, different risk

To isolate that point I made up two funds with exactly the same average daily return, 0.04%, but different standard deviations: 0.5% for the calm one, 1.5% for the wild one. Figure 1 draws a year of each.

Same average, but the wild fund ended lower (Table 4). One simulated year could be luck, but this gap is not: a fund that swings more grows slower on average, for a reason built into the arithmetic. Losses hurt more than equal gains help: lose 10% and then gain 10%, and you’re at 99%, not 100%. The more a fund swings, the more this “volatility drag” eats into growth. A rough rule, from the returns post: the yearly growth rate is about the average return minus half the variance.

From daily to yearly: why $\sqrt{252}$

There are about 252 trading days in a year. To turn daily numbers into yearly ones, with $\key{\mu}$ for the mean and $\risk{\sigma}$ for the standard deviation:

$$\key{\mu_{\text{annual}}} \approx 252 \times \key{\mu_{\text{daily}}}$$ $$\risk{\sigma_{\text{annual}}} \approx \sqrt{252} \times \risk{\sigma_{\text{daily}}}$$

The mean grows with the number of days, but the standard deviation grows with the square root. Why?

If days are independent, their variances add up. Over 252 days:

$$\risk{\text{Var}}(\text{year}) = 252 \times \risk{\text{Var}}(\text{day})$$ $$\risk{\text{SD}}(\text{year}) = \sqrt{252 \times \risk{\text{Var}}(\text{day})}$$ $$\risk{\text{SD}}(\text{year}) = \sqrt{252} \times \risk{\text{SD}}(\text{day})$$

Good days and bad days partly cancel out, so risk grows more slowly than time. I checked it by simulating 20,000 years of independent days with a 1% daily standard deviation. The standard deviation of the yearly totals came out at $\risk{15.86\%}$. The formula says $\sqrt{252} \times 1\% = 15.87\%$.

For our funds: the calm one is $0.5\% \times \sqrt{252} = 7.94\%$ a year, the wild one $\risk{23.81\%}$.

The same $\sqrt{252}$ shows up in the Sharpe ratio, which has its own post in this series.

The 68–95–99.7 rule, and why markets break it

If returns follow a normal distribution (the bell curve), the standard deviation tells you a lot (Table 5):

Real market returns are not quite normal. They have fat tails: extreme days happen far more often than the bell curve says. Take a fat-tailed distribution (Student’s t with 3 degrees of freedom, a standard bell-like curve with heavier ends) scaled to the same standard deviation, and ask how often a day lands more than 4 standard deviations from the mean (Table 6):

About 100 times more extreme days ($0.62 / 0.0063 \approx 97$). Simulating a million days of each gives the same picture. A risk model that assumes the bell curve will badly underestimate how often a crash day comes. The standard deviation is a good summary of everyday risk, and a poor guide to the worst days.

The code

The 2020 to 2025 numbers and the worked week. It reads the blog’s price file from the examples repo, so it runs as pasted (pip install numpy pandas). prices holds the five tickers’ daily closes, one column each; returns the daily returns; spy SPY’s column of them.

import numpy as np
import pandas as pd

URL = "https://raw.githubusercontent.com/ankurtrapasiya/examples/main/data/prices.csv"
prices = pd.read_csv(URL, index_col="date", parse_dates=True)   # adjusted closes, 5 tickers
returns = prices / prices.shift(1) - 1           # r_t = p_t / p_(t-1) - 1, one column per ticker

spy = returns["SPY"].dropna()                    # dropna: the first day has no day before it
mean = spy.mean()                                # 0.00065  (0.065% a day)
sd = spy.std()                                   # 0.0132   (1.32%); pandas divides by n - 1
sd_pop = spy.std(ddof=0)                         # divides by n, numpy's default
print(f"mean {mean:.5f}, sd {sd:.4f}, population sd {sd_pop:.4f}")

annual_mean = mean * 252                         # 16.4%
annual_sd = sd * np.sqrt(252)                    # 20.9%
growth = 100 * (1 + spy).prod()                  # $100 -> $228 over 2020-2025
print(f"annual mean {annual_mean:.1%}, annual sd {annual_sd:.1%}, $100 became ${growth:.0f}")

week = np.array([-0.8, 0.4, -0.3, 0.5, 0.7])     # the worked example, in %
print(week.mean().round(2), np.std(week, ddof=1).round(2), np.std(week).round(2))   # 0.1  0.63  0.56

It prints SPY’s daily mean and standard deviation (0.00065 and 0.0132; with over a thousand days, dividing by $n$ or $n - 1$ makes no visible difference), then the yearly figures (16.4%, 20.9%) and the $228, then the worked week: mean 0.1, sample standard deviation 0.63, population 0.56, where five days make the difference large.

Summary

Table 7 collects the formulas.

References

• Philip J. Romero and Tucker Balch, What Hedge Funds Really Do: An Introduction to Portfolio Management (Business Expert Press, 2014), chapter 2: returns, cumulative return, volatility
• Christopher M. Bishop and Hugh Bishop, Deep Learning: Foundations and Concepts (Springer, 2024), chapter 2, section 2.3: why dividing by $n$ underestimates variance (equation 2.60) and the $n - 1$ correction (2.63)
• Yves Hilpisch, Python for Finance: Mastering Data-Driven Finance (2nd ed., O’Reilly, 2018), chapter 8, “Financial Time Series”
• NumPy std, for the ddof setting