Machine learning for quantitative finance in ten ideas

Ten ideas, in the order each one needs the one before it. Same shape every time: the idea in one sentence, the formula with worked numbers, a real run where one helps, the trap, and where the idea shows up again. The real runs use the blog’s shared problem: predict next month’s volatility of a ticker from seven features computed from its own past prices (recent returns, recent volatility, distance from the moving average), five tickers, 3,685 training rows from 2019 to 2021 and 2,405 test rows from 2022 to 2023, all with scikit-learn. The full script is at the end of the post.

the quantity being estimated or chosenerror, riska bad outcomethe actual observed value

The target decides the problem

Full post: What is machine learning?.

Every target is a learnable rule plus noise nobody can predict:

What happens equals the pattern you can learn plus the luck you cannot. In finance $f$ is small and $\risk{\varepsilon}$ is big, so good models still look weak. The same rows can feed regression (the answer is a number), classification (the answer is a category) or clustering (no answer given); the target, not the rows, decides which. A classifier first estimates a probability, and the hard label is a second choice based on costs.

Expected error splits into three parts:

Bias is aiming at the wrong spot every time (the model is too simple). Variance is a shaky hand (the model changes with each training set). Noise is the floor; only new kinds of data lower it.

One big miss dominates the squared error. Eager learners (linear, tree) do the work at training time and can throw the data away; lazy learners (nearest neighbours) do nothing at training time and all the work at prediction time.

Real run. Next month’s volatility on the test months averages 22.8 points with a spread of 9.7. Predicting the training mean for every row scores a test RMSE of $\risk{9.84}$. That number is the bar every model below has to clear, and some of them do not.

The data is part of the model

Full post: The data is part of the model.

Which rows you keep, fill, order or reuse changes results as much as the algorithm does.

The split here is in time: train on 2019 to 2021, test on 2022 to 2023, never shuffled. Consecutive days share most of a month, so a random split would put near-twins of every test row into the training set and flatter every model. For a time series the test must be later than the training, and a walk-forward loop (fit on a window, test on the next, slide) is the gold standard.

Why data goes missing matters: missing completely at random (an API glitch) only shrinks the sample; missing at random (small companies report less often) needs care; missing because of the value (no volume because nothing traded, no earnings because bankrupt) is the signal itself and must not be deleted or mean-filled. The price rule: fill forward, then fill backward only where nothing came before. Keep a missingness indicator column.

Squared error chases outliers; absolute loss, Huber loss or a tree (only the side of a split matters) resist them. Every data-dependent step is training: scaling, filling, feature selection, choosing settings. Fit them on the training part only.

Every feature has a timestamp

Full post: Every feature has a timestamp.

For every feature ask one question: when did this value become knowable, and could I compute it live? Leakage is a feature that secretly contains the answer, through time (it uses the future) or through pooling (a statistic computed over train and test together).

Real run, a leak on purpose. The seven honest features are all computed from prices up to the day of the forecast. I added an eighth: SPY’s volatility over the same month the model is trying to predict. It is not knowable in advance, and it would be an easy mistake to make when joining a market-wide table onto a per-ticker one.

The leak looks like a better model. Nothing in the score says otherwise. The first defence is to lag every input to the previous close; the second is to rerun with the last few days removed and see whether earlier predictions change.

Scaling changes influence, not information. Distance-based models need it; trees do not, because only order matters. Correlated twin features make linear coefficients swing (their sum stays stable) and split a forest’s importance between them.

Rolling z-score, worked: window mean 100, σ 2, price 106 gives $z = 3$; in a crash the window σ jumps to 10 and the same six-point gap reads $z = 0.6$. The z-score hides the crash.

A linear model is a weighted sum

Full post: Linear regression.

Start from a base number and add a fixed amount per unit of each feature, like a taxi fare. Worked: $\hat{y} = 0.001 + 0.5 \times \text{momentum} - 0.2 \times \text{vol}$ with momentum 0.02 and vol 0.01 gives $0.001 + 0.010 - 0.002 = 0.009$. Each $\beta$ is the change in prediction per unit of its feature holding the others fixed; only standardised coefficients can be compared by size.

Fitting is least squares: append a column of ones so the solve finds an intercept, then np.linalg.lstsq, or LinearRegression() in scikit-learn, which does the same.

Real run, when a line wins. A made-up dataset: 1,000 rows, three features uniform on −1 to 1, $y = 50x_1 + 30x_2 - 20x_3 + \text{small noise}$. Out of sample:

A tree cannot draw a slope. It builds a staircase of constants and cannot extrapolate past the training range, so on a smooth linear surface it is a hundred times worse. Polynomial terms make curves but the model is still linear in $\beta$. Logistic regression squashes the weighted sum into a probability; its boundary is still a straight line. Read the residuals: a U-shape means a missed curve, runs of the same sign over time mean serial correlation, a widening fan means uneven noise.

$R^2 = 1 - \text{RSS}/\text{TSS}$ is “how much better than predicting the mean”; in finance a value well below 0.1 can be real. A coefficient’s $t = \hat\beta / \text{SE}$ with $|t| \gtrsim 2$ is the usual bar.

Nearest neighbours store everything

Full post: Nearest neighbours.

Predict what similar past cases did: keep all the data, and at query time find the $k$ closest rows and average them (regression) or vote (classification).

The dial runs backwards: raising $k$ makes the model simpler, the opposite of tree depth or polynomial degree. Effective capacity is about $N/k$.

The last row is the curse of dimensionality: in many dimensions everything is far from everything, and a “neighbourhood” is almost the whole space. Scaling fixes units, not this; drop irrelevant features or weight each one. Cost: training is free, each prediction scans $N \times p$ numbers. Daily allocation on 7,500 rows is trivial and explainable (“today looks like these five dates”); a high-frequency book at 500,000 rows per tick is not.

A tree asks one question at a time

Full post: Decision trees.

A tree is a chain of yes/no questions, one feature and one threshold each, ending in a leaf that predicts a constant. It grows greedily: at each node try every feature and threshold, keep the best split, never look back.

A regression tree needs three decisions, and the classic recipe makes them like this:

Growing is recursion: cut, then grow a tree on each side. Prediction walks from the root, answering one question per level, until it reaches a leaf.

Real run, when a tree wins. The same made-up setup, but $y$ is a sum of three step functions:

Out of sample:

Three thresholds are three questions. A line has to average across each step and misses on both sides of it. Trees capture thresholds and interactions without hand-made features, need no scaling, and are unstable: a small change in the data can flip the root split and everything below, which is the reason to bag them.

Impurity, for classification trees, measures how mixed a node is:

Worked regression split: targets {2, 4, 9, 11}. Cut A into {2, 4} and {9, 11}: means 3 and 10, squared error 2 + 2 = 4. Cut B into {2, 9} and {4, 11}: means 5.5 and 7.5, error 24.5 + 24.5 = 49. Cut A wins. Gini gain: a 50/50 parent (0.5) split into two 80/20 halves (0.32 each) gains 0.18.

Averaging helps only when the errors differ

Full post: Bagging.

Bagging trains many copies of a model, each on a bootstrap sample (draw $N$ rows with replacement), and averages them. Chance a row is never drawn: $(1 - 1/N)^N \to 1/e \approx 37\%$, so each copy sees about 63% of distinct rows and the rest are out-of-bag, a free validation set.

Averaging shrinks only the part of the errors that differs from copy to copy. The shared part $\key{\rho}\risk{\sigma^2}$ stays however many copies you add, like polling a hundred people who all read the same newspaper. Random forests attack $\key{\rho}$ by letting each split choose from a random subset of features; boosting trains copies one after another on the errors so far and cuts bias instead.

Real run, bags of fully grown trees. BaggingRegressor around a tree with one row per leaf, test RMSE on the 2022 to 2023 months:

The first twenty trees take a third off the error; the next eighty take nothing. That levelling is the $\key{\rho}\risk{\sigma^2}$ floor. Twenty bagged trees also flatten the leaf-size curve: a single tree’s test error runs from 15.7 down to 8.4 across leaf sizes, the bagged version from 10.6 to 8.1.

Real run, random splits. splitter="random" picks the cut point at random instead of searching for the best one. On this noisy target it trained three to seven times faster and, for leaf sizes above a handful, scored better than the searching tree (10.24 against 12.01 at leaf 10, 9.22 against 10.25 at leaf 30). The search was finding cuts that fit the training noise. Twenty bagged random trees with one row per leaf scored 9.80, against 10.61 for the searching kind. A random tree is weaker in principle and cheaper in practice, and on noisy data the “weakness” is a form of regularisation; that is the trade random forests and extra-trees exploit.

Bagging helps unstable learners. A bag of linear regressions returns almost exactly the single regression’s answer, because a straight line barely moves between resamples and there is nothing to average away.

A score needs a split, a metric and a baseline

Full post: Scoring a model.

For a time series the test must be later than the training. The three error metrics on misses 1, 1, 1, 5: MAE 2, MSE 7, RMSE 2.65. RMSE is at least MAE, equal only when every miss is the same size; a big gap means a few large misses.

Real run. The baseline from section 1, predicting the training mean, scores 9.84. A fully grown tree scores 15.69, worse than guessing the average. Twenty bagged trees score 10.61, still worse. A single tree with 200 rows per leaf scores 8.41 and the plain linear model 7.50. Every one of those numbers is meaningless until it sits next to 9.84, and the first two are the kind of result that gets reported as “the model works” when nobody checked the baseline. For classification the bar is the majority class: 58% accuracy when 60% of days are up is worse than doing nothing.

Classification metrics from a confusion matrix: sensitivity (recall) divides true positives by the actual positives; specificity divides true negatives by the actual negatives; precision divides true positives by the predicted positives. In a credit example with 10,000 people and 333 defaulters, total error 2.75% looked fine while 252 of 333 defaulters were missed. Moving the threshold from 0.5 to 0.2 cut the misses to 138 at the cost of more false alarms. ROC plots the true positive rate against the false positive rate across every threshold; AUC 0.5 is a coin flip.

The gap between seen and unseen is the whole game

Full post: Bias and variance.

Training error always falls as the model gets more flexible. Validation error falls, bottoms out, then rises. Stop at the bottom.

Real run. One regression tree, min_samples_leaf swept from 1 to 200:

At leaf size 1 the tree has one row per leaf and reproduces the training targets; its test error is 60% worse than guessing the mean. As leaves grow the tree averages more rows, training error rises and test error falls, and on this target the bottom is at 200 rows per leaf or beyond: next month’s volatility is so noisy that a tree needs to average a lot of months before it says anything true. (The classic U-shape, where test error turns back up, appears on less noisy targets; here the right end of the sweep is where it starts.) Diagnose from the two numbers:

More data helps variance only. Complexity dials: tree depth and leaf size, $k$ (reversed), polynomial degree, λ (reversed), boosting rounds. Cross-validation averages the error over $K$ folds; for a time series use forward folds, because random folds leak the future. Choose the simplest model within one standard error of the best.

Model choice is a systems decision

Full post: Choosing a model.

Real run, the lessons in numbers. The linear surface: linear 0.06, tree 7.79. The three steps: tree 0.21, linear 2.52. No method is best on every dataset. On the real volatility target the plain linear model (7.50) beat every tree and every bag, because the signal is weak and nearly linear, and the cheapest model had the least room to fit noise. A random-split tree trained three to seven times faster than a searching one. Twenty bagged trees cost twenty fits and twenty queries for a third off a fully grown tree’s error. A thousand-row, twenty-feature table costs a KNN 20,000 numbers per prediction and a linear model 20.

Interpretability can be a binding constraint, and accepting a little less accuracy is then correct. Tail latency (P99), not the mean, decides whether a deadline is met. More history helps only if it is relevant. A small observed lead can be noise, and reliability can beat a marginal gain.

The formula sheet

The arithmetic to be fluent in

1. MAE, MSE and RMSE from a list of misses.
2. Gini and entropy from class fractions; information gain in bits.
3. The bootstrap 63/37 split and the $\rho\sigma^2$ floor.
4. The reversed dials: small $k$ is complex, big λ is simple.
5. Bayes with counts: 1% base rate, 90% sensitivity, 3% false alarms gives 90/(90 + 297) = 23%.
6. Sensitivity, specificity and precision from a confusion matrix.
7. Read a train-test pair: 0.5 and 2.0 is overfit; 1.9 and 2.0 is underfit.

The script

Every number and chart above comes from one script, which runs in a few seconds with uv run (the dependencies are declared at the top of the file). It uses the blog’s shared price file and two small helpers for loading prices and drawing charts.

# /// script
# requires-python = ">=3.11"
# dependencies = ["numpy", "pandas", "matplotlib", "scikit-learn"]
# ///
"""Numbers and chart for the post "Machine learning for quantitative finance in ten ideas".

Everything runs on the blog's shared market data (examples/market_data.py: SPY, AAPL, JPM, XOM,
GLD, 2019-2023, seven price-derived features, target = next month's volatility) with scikit-learn,
plus two tiny made-up datasets that favour a line or a tree.
Run:  uv run examples/machine-learning-in-ten-ideas/ten_ideas.py
"""
import sys
import time
from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
from sklearn.ensemble import BaggingRegressor
from sklearn.linear_model import LinearRegression
from sklearn.tree import DecisionTreeRegressor

sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
import chartstyle as cs  # noqa: E402
import market_data as md  # noqa: E402

OUT = Path(__file__).resolve().parents[2] / "public" / "posts" / "machine-learning-in-ten-ideas"


def rmse(y, yhat):
    return float(np.sqrt(np.mean((y - yhat) ** 2)))


def p(label, value):
    print(f"{label:<64} {value}")


df = md.features()
train, test = md.split(df)                     # 2019-2021 train, 2022-2023 test, in time order
Xtr, ytr = train[md.FEATURES].to_numpy(), (train["next_vol"] * md.ANNUALISE).to_numpy()
Xte, yte = test[md.FEATURES].to_numpy(), (test["next_vol"] * md.ANNUALISE).to_numpy()
print("\n== data ==")
p("training rows / test rows", f"{len(ytr)} / {len(yte)}")
p("target: next month's volatility, % a year; test mean / sd", f"{yte.mean():.1f} / {yte.std():.1f}")
p("baseline: predict the training mean, test RMSE", f"{rmse(yte, np.full_like(yte, ytr.mean())):.2f}")

print("\n== 9. one tree, leaf size swept ==")
leaves = list(range(1, 31)) + [40, 50, 75, 100, 150, 200]
tree_tr, tree_te, tree_t = [], [], []
for ls in leaves:
    m = DecisionTreeRegressor(min_samples_leaf=ls, random_state=0)
    t0 = time.perf_counter(); m.fit(Xtr, ytr); tree_t.append(time.perf_counter() - t0)
    tree_tr.append(rmse(ytr, m.predict(Xtr))); tree_te.append(rmse(yte, m.predict(Xte)))
for ls, a, b in zip(leaves, tree_tr, tree_te):
    if ls in (1, 5, 10, 20, 30, 50, 100, 200):
        p(f"leaf size {ls:3}: train / test RMSE", f"{a:.2f} / {b:.2f}")
best = int(np.argmin(tree_te))
p("best test RMSE", f"{tree_te[best]:.2f} at leaf size {leaves[best]}")

print("\n== 7. twenty bagged trees, same sweep ==")
bag_tr, bag_te = [], []
for ls in leaves:
    m = BaggingRegressor(DecisionTreeRegressor(min_samples_leaf=ls), n_estimators=20, random_state=0, n_jobs=-1)
    m.fit(Xtr, ytr)
    bag_tr.append(rmse(ytr, m.predict(Xtr))); bag_te.append(rmse(yte, m.predict(Xte)))
for ls, a, b in zip(leaves, bag_tr, bag_te):
    if ls in (1, 5, 10, 20, 30, 50, 100, 200):
        p(f"leaf size {ls:3}: bagged train / test RMSE", f"{a:.2f} / {b:.2f}")
p("bagged test RMSE range over all leaf sizes", f"{min(bag_te):.2f} to {max(bag_te):.2f}")
for b_ in [1, 5, 10, 20, 50, 100]:
    m = BaggingRegressor(DecisionTreeRegressor(min_samples_leaf=1), n_estimators=b_, random_state=0, n_jobs=-1).fit(Xtr, ytr)
    p(f"bags {b_:3} of fully grown trees: test RMSE", f"{rmse(yte, m.predict(Xte)):.2f}")

print("\n== 7. random splits against chosen splits ==")
rt_te, rt_t = [], []
for ls in leaves:
    tes, ts = [], []
    for seed in range(10):
        m = DecisionTreeRegressor(min_samples_leaf=ls, splitter="random", random_state=seed)
        t0 = time.perf_counter(); m.fit(Xtr, ytr); ts.append(time.perf_counter() - t0)
        tes.append(rmse(yte, m.predict(Xte)))
    rt_te.append(float(np.mean(tes))); rt_t.append(float(np.mean(ts)))
for ls in (1, 10, 30):
    i = leaves.index(ls)
    p(f"leaf size {ls:3}: chosen vs random split test RMSE", f"{tree_te[i]:.2f} vs {rt_te[i]:.2f}; fit {1000*tree_t[i]:.1f} vs {1000*rt_t[i]:.1f} ms")
m = BaggingRegressor(DecisionTreeRegressor(min_samples_leaf=1, splitter="random"), n_estimators=20, random_state=0, n_jobs=-1).fit(Xtr, ytr)
p("20 bagged random-split trees, leaf 1: test RMSE", f"{rmse(yte, m.predict(Xte)):.2f}")

print("\n== 4 and 6. a line or a tree, on data that favours each ==")
rng = np.random.default_rng(0)
X = rng.uniform(-1, 1, (1000, 3))
y_line = 50 * X[:, 0] + 30 * X[:, 1] - 20 * X[:, 2] + rng.uniform(-0.1, 0.1, 1000)
y_step = 8 * (X[:, 0] > 0) + 5 * (X[:, 1] > 0) + 3 * (X[:, 2] > 0) + 0.5 * X[:, 2]
idx = rng.permutation(1000); tr_i, te_i = idx[:600], idx[600:]
for name, y in [("linear surface", y_line), ("three steps", y_step)]:
    lin = LinearRegression().fit(X[tr_i], y[tr_i]); tr = DecisionTreeRegressor(min_samples_leaf=1, random_state=0).fit(X[tr_i], y[tr_i])
    p(f"{name}: linear / tree test RMSE", f"{rmse(y[te_i], lin.predict(X[te_i])):.2f} / {rmse(y[te_i], tr.predict(X[te_i])):.2f}")

print("\n== 3. the leak: a same-day feature ==")
# add SPY's same-day next-month volatility as a feature for the other tickers: not knowable in advance
spy = df.xs("SPY", level="ticker")["next_vol"] * md.ANNUALISE
leak = df.join(spy.rename("spy_same_month"), on="date")
ltr, lte = md.split(leak)
m = LinearRegression().fit(ltr[md.FEATURES], ltr["next_vol"] * md.ANNUALISE)
p("linear, honest features: test RMSE", f"{rmse(lte['next_vol'] * md.ANNUALISE, m.predict(lte[md.FEATURES])):.2f}")
m = LinearRegression().fit(ltr[md.FEATURES + ['spy_same_month']], ltr["next_vol"] * md.ANNUALISE)
p("linear, plus a same-month feature (leak): test RMSE", f"{rmse(lte['next_vol'] * md.ANNUALISE, m.predict(lte[md.FEATURES + ['spy_same_month']])):.2f}")

# ------------------------------------------------------------------ chart ----
cs.apply()
fig, (a1, a2, a3, a4) = plt.subplots(4, 1, figsize=(8, 3.8 * 4))
cs.title(a1, "One regression tree: error on seen and unseen months as leaves grow", 1)
a1.plot(leaves, tree_tr, color=cs.BLUE, marker="o", ms=3, label="training months (seen)")
a1.plot(leaves, tree_te, color=cs.MAROON, marker="o", ms=3, label="test months (unseen)")
a1.set_xscale("log"); a1.set_xlabel("Leaf size (minimum rows in a leaf, log scale)"); a1.set_ylabel("RMSE (volatility points)")
a1.legend()
cs.title(a2, "Twenty bagged trees: the test curve goes flat", 2)
a2.plot(leaves, bag_tr, color=cs.BLUE, marker="o", ms=3, label="training months, bagged")
a2.plot(leaves, bag_te, color=cs.MAROON, marker="o", ms=3, label="test months, bagged")
a2.plot(leaves, tree_te, color=cs.GREY, lw=1, label="test months, one tree (from panel 1)")
a2.set_xscale("log"); a2.set_xlabel("Leaf size (minimum rows in a leaf, log scale)"); a2.set_ylabel("RMSE (volatility points)")
a2.legend()
cs.title(a3, "Random splits against chosen splits: error on the test months", 3)
a3.plot(leaves, tree_te, color=cs.MAROON, marker="o", ms=3, label="tree that searches for the best split")
a3.plot(leaves, rt_te, color=cs.BLUE, marker="o", ms=3, label="tree that splits at random (mean of 10)")
a3.set_xscale("log"); a3.set_xlabel("Leaf size (minimum rows in a leaf, log scale)"); a3.set_ylabel("RMSE (volatility points)")
a3.legend()
cs.title(a4, "Time to build the tree", 4)
a4.plot(leaves, [1000 * t for t in tree_t], color=cs.MAROON, marker="o", ms=3, label="tree that searches for the best split")
a4.plot(leaves, [1000 * t for t in rt_t], color=cs.BLUE, marker="o", ms=3, label="tree that splits at random (mean of 10)")
a4.set_xscale("log"); a4.set_xlabel("Leaf size (minimum rows in a leaf, log scale)"); a4.set_ylabel("Training time (milliseconds)")
a4.legend()
cs.save(fig, OUT / "leaf-size-sweep.svg")