Backtesting in Python

A trading rule is a hypothesis: “when the 20-day average price crosses above the 100-day average, the market is turning up, so buy.” A backtest is the experiment that checks it. You take years of past prices, apply the rule day by day as if you had been there, keep the books (cash, position, profit and loss), and read off the result at the end. A systematic strategy, from a hedge fund’s model to a retail investor’s moving-average rule, goes through this step before it touches money, because the alternative is finding out with real money.

The result is a number, and people believe numbers from a backtest too easily. This post explains what a backtest is made of, builds the simplest honest one in a few lines of pandas, runs the same rule through two Python libraries, and uses the differences between the answers to show where backtests go wrong. The last section is a guide to the libraries worth knowing.

What a backtest is made of

Five parts, whatever the library:

• Historical prices. Daily bars with open, high, low, close and volume, or finer. The quality of the data bounds the quality of the test: prices must be adjusted for splits and dividends, and the set of tickers must be the one that existed then, not the survivors.
• The rule. A function from everything known up to today to a position for tomorrow. Moving-average crossovers, momentum, mean reversion, a machine-learning model’s forecast.
• The execution model. The part beginners skip and professionals spend most of their time on. When does the order fill, at what price, and what does it cost in commission, bid-ask spread, slippage (the price moving against you between decision and fill) and market impact (your own order moving the price)?
• Portfolio accounting. Cash, shares held, mark-to-market value (what the holdings are worth at that day’s prices), every day.
• Evaluation. Total return, Sharpe ratio, maximum drawdown (the worst peak-to-trough fall), number of trades, win rate, time in the market.

There are two ways to build the machine. An event-driven backtester walks through time one bar at a time, calling your code at each step with only the data up to that point, exactly as a live trading system would; it is realistic, it can model partial fills and queued orders, and it is slow. A vectorised backtester computes the signal for every day at once as array operations and multiplies positions by returns; it is far faster, which makes it the tool for parameter sweeps, and it is easy to leak tomorrow’s data into today’s decision because all of time is in one array. The example below uses both.

One rule by hand: the look-ahead trap

The rule: hold SPY when its 20-day average close is above its 100-day average, hold cash otherwise. One formula says everything a vectorised backtest does:

s, the position (1 = holding SPY, 0 = cash)

The return the rule earns on day $t$ is the market’s return $r_t$ times the position decided the day before. The subscript is the point. The position on day $t$ is computed from day $t$‘s close, so it cannot earn day $t$‘s return; the earliest it can act is day $t+1$. Write $\key{s_t} \, r_t$ instead and the backtest trades on information it did not have yet. That is look-ahead bias, and in a vectorised backtest it is one missing .shift(1).

The three code blocks below run as pasted, in order, in one Python session (pip install pandas numpy backtesting vectorbt). This one reads the blog’s price file from the examples repo; spy holds SPY’s daily closes, ret its daily returns, and in_market the rule’s position, one per day.

import pandas as pd

URL = "https://raw.githubusercontent.com/ankurtrapasiya/examples/main/data/prices.csv"
prices = pd.read_csv(URL, index_col="date", parse_dates=True)
spy = prices["SPY"]                      # adjusted daily closes, 2020-2025
ret = spy.pct_change().fillna(0.0)       # daily returns; the first day earns nothing
fast, slow = spy.rolling(20).mean(), spy.rolling(100).mean()
in_market = (fast > slow).astype(int)    # 1 = hold SPY, 0 = hold cash

cheat = in_market * ret                  # set on the same close: impossible
honest = in_market.shift(1) * ret        # position set on the day before
total = lambda r: (1 + r).prod() - 1     # multiply the growth factors
print(f"buy and hold {total(ret):+.1%}, cheat {total(cheat):+.1%}, honest {total(honest):+.1%}")

It prints +128.4%, +113.6% and +106.7%, the first column of Table 1.

The cheating version is not a wild number. It is $\key{6.9}$ percentage points better than the honest one, which is exactly the kind of gap that makes a mediocre rule look like a good one. The reason is mechanical: a crossover rule buys after a rise has started and sells after a fall has started, so every entry and exit is late by construction. Shifting the position by one day makes it one day later still, and the days just after a crossing are the days the trend is strongest.

Note. A look-ahead bug never announces itself. The code runs, the chart looks plausible, the numbers are a bit too good. The test is to ask, for every input to the rule on day $t$, “was this number printed before the close of day $t$?” If any answer is no, the result is fiction.

Panel 1 of Figure 1 shows the rule’s eleven position changes over 2020 to 2025. The signal is slow by design and needs 100 days of history before it says anything, so it was in cash through the March 2020 crash and the first two months of the recovery by warm-up, not by judgement; its first buy is 3 June 2020. Panel 2 is the money: the library version of the rule in the next section (next-day fills, 0.1% commission) turned $\usd 10{,}000$ into about $\usd 19{,}100$, while buy and hold reached $\usd 22{,}800$. On SPY over 2020 to 2025, the crossover rule lost to doing nothing, and its worst fall was −19.2% (December 2022) against −33.7% (March 2020, which it missed only because its averages did not exist yet); that is the trade it offers.

The same rule in a library: the fill-timing trap

backtesting.py is the smallest full backtester in Python: event-driven, one instrument, a Strategy class with two methods, a Backtest that runs it, and a statistics table at the end. The crossover rule is eight lines.

from backtesting import Backtest, Strategy
from backtesting.lib import crossover

class SmaCross(Strategy):
    fast, slow = 20, 100

    def init(self):                       # once, before the first bar
        sma = lambda s, n: pd.Series(s).rolling(n).mean()
        self.fast_ma = self.I(sma, self.data.Close, self.fast)
        self.slow_ma = self.I(sma, self.data.Close, self.slow)

    def next(self):                       # once per bar, data up to that bar
        if crossover(self.fast_ma, self.slow_ma):
            self.buy()
        elif crossover(self.slow_ma, self.fast_ma):
            self.position.close()

# the library wants Open, High, Low and Close columns; with closes only, all four are the close
ohlc = pd.DataFrame({c: spy for c in ["Open", "High", "Low", "Close"]})
bt = Backtest(ohlc, SmaCross, cash=10_000, commission=0.001,
              exclusive_orders=True, finalize_trades=True)
stats = bt.run()
print(stats[["Return [%]", "Sharpe Ratio", "Max. Drawdown [%]", "# Trades"]])   # 91.4, 0.86, -24.3, 6

It prints the first four rows of Table 2. self.I(...) registers an indicator so the library can show it on its chart and hand your code only its values up to the current bar; exclusive_orders=True makes each new order close the open position first.

The hand version said $\key{+106.7\%}$. The library says $\key{+91.4\%}$. Same rule, same data, same years. Three conventions explain the gap, none of them a bug:

1. The library waits for a real crossing. If the fast average already sits above the slow one on the first day the slow one exists, the hand version is “in the market” from that day, while the library’s crossover needs the fast line to actually cross. On this data it changes nothing: the 100-day average first exists at the end of May 2020, when the fast line was still below it after the March crash, so both versions start with the crossing on 3 June 2020. On data that starts in a rising market it skips a season of gains.
2. The library fills one day later. A signal on day $t$‘s close becomes an order that fills at the next bar’s open. With daily closes only (the example sets open equal to close), that is the close of day $t+1$, and the position earns from day $t+2$. The hand version earned from day $t+1$.
3. Commission. 0.1% of every fill, $\usd 166$ over six round trips. Small here; for a rule that trades daily it is everything.

Put the extra day of delay into the hand version and it reports $\key{+94.2\%}$; the rest is the commission. Neither number is the right one. What matters is that a single day of fill delay moved the result from 107% to 94%, and a strategy whose backtest depends that much on when the fill is assumed to happen is a strategy you do not understand yet. Every library makes these choices for you; read the defaults before you believe the output.

Searching for parameters: the over-fitting trap

Why 20 and 100? Only because they are round numbers. The natural next step is to try every pair and keep the best, which is the job vectorised backtesting is built for. vectorbt runs every parameter combination as one wide array, using NumPy and Numba under the hood. The 1,225 fast/slow pairs on four years of SPY below took 9 seconds on the first call, almost all of it Numba compiling, and 0.12 seconds on the second.

import numpy as np, vectorbt as vbt

train, test = spy[:"2023"], spy["2024":]  # choose on 2020-2023, check on 2024-2025
windows = np.arange(5, 255, 5)            # 50 windows, 1,225 fast/slow pairs

def sweep(px):                            # every pair's total return on one price series
    fast_ma, slow_ma = vbt.MA.run_combs(px, window=windows, r=2, short_names=["fast", "slow"])
    pf = vbt.Portfolio.from_signals(px, fast_ma.ma_crossed_above(slow_ma),
                                    fast_ma.ma_crossed_below(slow_ma), fees=0.001, freq="1D")
    return pf.total_return()              # one number per (fast, slow) pair

in_sample, out_of_sample = sweep(train), sweep(test)
best = in_sample.idxmax()
print(best, f"{in_sample[best]:+.1%} then {out_of_sample[best]:+.1%}")   # (80, 140) +71.9% then +7.6%
print(round(in_sample.corr(out_of_sample, method="spearman"), 2))       # -0.07: no relation

It prints the best in-sample pair, (80, 140), with its +71.9% and then +7.6% on the unseen years, the top row of Table 3, and the rank correlation of −0.07 between the two periods across all 1,225 pairs. The first run takes several seconds while Numba compiles; later runs take a fraction of a second.

The sweep is run on 2020 to 2023 only. The pairs are then scored on 2024 to 2025, which the search never saw: the same discipline as a train/test split in machine learning, for the same reason.

Panel 3 of Figure 1 plots all 1,225 pairs, in-sample return across, out-of-sample return up. It is a cloud with no slope: the rank correlation between the two is $\key{-0.07}$. The best tenth of pairs in-sample went on to earn a median $\key{+8.0\%}$; the worst tenth earned $\key{+15.9\%}$. The search found nothing real. It found the pairs that happened to fit 2020–2023 (a crash, a recovery, a bear year), and 2024–2025 was a different market, a steady rise of +47.4% in which not one of the 1,225 pairs beat buy and hold.

Note. Picking the best of 1,225 backtests and reporting its return is not a backtest of a strategy. It is a backtest of your search, and the expected result of searching is to find luck. The honest number is the one from data the search never touched, and even that gets used up: look at the test set twice and it has become training data.

There is a human side to this. A backtest reads like a record of what happened, and we treat records as facts. The search that produced it is not in the chart. So “it backtested at 70%” persuades people who would never believe “I tried 1,225 things and this one worked”, even though it is the same claim.

Habits that keep a backtest honest

• Lag the signal. The position on day $t$ comes from data up to day $t-1$. In pandas that is .shift(1); in an event-driven library it is the default, which is a reason to use one.
• Charge for every fill. Commission, spread and slippage, with a pessimistic guess if you have no better one. A rule that only works without costs does not work.
• Hold out time. Choose parameters on one period, report on a later one. For a strategy you will keep re-tuning, use walk-forward testing: fit on a window, test on the next, slide, repeat, and report only the stitched-together test periods.
• Count the trials. Keep a note of how many variants you tried, and treat the best one’s result with the suspicion that number deserves.
• Use the data that existed. Tickers that were later delisted belong in the test; a universe built from today’s survivors backtests beautifully.

Costs, sizing, capacity and the benchmark

Four more checks that decide whether a rule that passed the backtest is a rule to run.

Costs decide the ranking. The crossover above changed position 11 times in 2020 to 2025. A rule that holds SPY only on the day after an up day changes position 747 times. Charge both the same cost per position change (Table 4):

The slow rule gives up eleven points; the busy one is destroyed by a tenth of a percent. The arithmetic is turnover times cost: 40% of the portfolio bought or sold daily at 0.10% is 10% a year before any edge is counted. Decide on net returns; the post on trading costs has the full model.

Size by risk, not by conviction. Two positions with the same 2% forecast and volatilities of 10% and 40% get sizes in the ratio 4 to 1, the riskier one smaller, so that each contributes about the same risk. A rule that sizes every trade the same lets its wildest names set its drawdown.

Every strategy has a capacity. An edge of 0.50% a trade against an impact cost of 0.10%, 0.30% and 0.70% as the order size grows is a net 0.40%, 0.20% and −0.20%. Past some size the rule’s own trading moves the price enough to eat the edge, and the backtest, which assumed a fixed price, never saw it.

The benchmark must match the mandate. A long-only equity rule is judged against a broad equity index over the same dates, not against cash: beating cash in a bull market is beta, not skill. The crossover here lost to buy and hold by 22 points; “it made +107%” is the wrong sentence, and “it made 22 points less than doing nothing, with a shallower drawdown” is the right one.

The libraries

Six names come up. The short list is two.

backtesting.pyOne instrument, a rule in a few lines, a statistics table and an interactive chart

Whythe smallest complete event-driven backtester: Strategy with init and next, a Backtest that runs it, built-in parameter optimisation with heatmaps; built on pandas, NumPy and Bokeh; actively released

Move on whenyou need a portfolio of many instruments, or you want to sweep thousands of parameters

vectorbtThousands of parameter combinations, many assets, in seconds

Whyvectorised on NumPy and Numba (with an optional Rust engine), so a whole grid of strategies is one array operation; rich portfolio statistics; the open-source community edition of a commercial product, under Apache 2.0 with a Commons Clause (you may not sell it as a product)

Move on whenthe rule needs path-dependent logic that does not vectorise, or you need realistic order handling

BacktraderA full event-driven platform: multiple data feeds, timeframes, brokers, sizers, analysers

Whythe most complete pure-Python engine, written by Daniel Rodriguez from 2015, with live-trading adapters for Interactive Brokers and Oanda and a large archive of answered questions

Move on whenyou need maintenance: the last release on PyPI is from April 2023 and the repository last changed in 2024, so it runs on today's Python by goodwill, not by a release cycle

Zipline (reloaded)Research over a universe of stocks with factor models and a point-in-time database

Whythe event-driven engine Quantopian built and ran its crowd-sourced fund on, until it closed in late 2020; now the zipline-reloaded fork maintained by Stefan Jansen for the readers of his book; initialize and handle_data callbacks, data bundles you ingest once, and the Pipeline API for computing factors across every asset on every day

Move on whenyou want one ticker and a quick answer; the data ingestion alone is an afternoon

LEANInstitutional-grade: many asset classes, C# or Python, the same code for backtest and live

WhyQuantConnect's open-source engine (Apache 2.0), event-driven, with brokerage integrations; the lean CLI runs backtests, research notebooks and live trading locally in Docker

Move on whenyou are not planning to trade live through it: the engine is heavy, and the data model assumes QuantConnect's formats

PyAlgoTradeHistorical interest only

Whyan early, readable event-driven library; its README now says it is deprecated and no longer maintained, the last release was in 2018, and its author's successor project is Basana, an async framework aimed at crypto exchanges

Move on whennow; read it to learn how an event loop, a broker and a strategy fit together, then use something maintained

The practical pattern: prototype in pandas, where you can see every line and there is nothing to misunderstand; sweep in vectorbt when there are parameters to search; confirm in an event-driven engine (backtesting.py for one instrument, LEAN or Backtrader for a portfolio) so the fill timing and costs are modelled the way a live system would. If the three disagree, as they did above, that disagreement is what you have learned.

Where this shows up at work

A backtest is the same object as a replay test in a settlement or payments system: historical inputs, the new logic, a diff against what actually happened. The traps are the same too. Using a field that was back-filled later is look-ahead. Tuning a fraud threshold on last month’s cases and reporting its catch rate on last month’s cases is in-sample. Replaying only the transactions that completed is survivorship. If you have built a replay harness you already know how to read a backtest, and when to distrust one.

The script behind all the numbers

Every table and the chart come from examples/backtesting/sma_crossover.py. To run it, save it next to the shared helpers market_data.py and chartstyle.py and the price file data/prices.csv, keeping the folders as in the links, then run uv run examples/backtesting/sma_crossover.py.

Summary

Table 5 is the checklist.

References

• Philip J. Romero & Tucker Balch, What Hedge Funds Really Do (Business Expert Press, 2014), ch. 2 on returns and risk measures; ch. 12 on adjusting prices for splits and dividends and on the survivor bias from delisted tickers; ch. 14 on “peeking” into the future, the data-mining fallacy and cross validation.
• Marcos López de Prado, Advances in Financial Machine Learning (Wiley, 2018), ch. 11 “The Dangers of Backtesting”, on backtest over-fitting and counting trials (outside the usual reading list on this site, cited because it covers the sweep above).
• Library documentation, read for the cards above: backtesting.py, vectorbt, Backtrader, Zipline reloaded, LEAN and PyAlgoTrade.