Checklist, in Backtesting & Validation
Backtest Hygiene Checklist
Hygiene comes before validation. A backtest can pass an out-of-sample test and still be worthless if the underlying simulation leaks the future or ignores costs. Walk this list once when you build the harness, then re-confirm it whenever you add a new data source or feature.
12 items. Published May 26, 2026. By AI Fin Hub Research.
Your progress
Move item by item and keep your place
Progress saves locally, so you can work through the page over multiple sessions without resetting your checklist.
Sections
Work in focused batches instead of one long wall
Section 1
Phase 1: Data integrity
Section 2
Phase 2: Look-ahead and alignment
Section 3
Phase 3: Cost realism
Section 4
Phase 4: Reproducibility
Tips
Small moves that make the checklist easier to finish
Run the numbers next
- Playgrounds Walk-Forward Validator
Upload a returns CSV. Rolling or expanding IS/OOS windows, per-window Sharpe, walk-forward efficiency, and a concatenated OOS equity curve. Catches regime.
- Calculators Backtest Overfitting Score
Upload a backtest trade log and compute Probability of Backtest Overfitting (PBO), Deflated Sharpe Ratio, and the odds your edge survives live trading.
- Generators Synthetic Market Data Generator
Generate synthetic price series — geometric Brownian motion, GARCH(1,1) with volatility clustering, regime-switching bull/bear, or copula-linked.
Questions people ask next
What is the most common cause of look-ahead bias in backtests?
Entering a position on the same bar that generated the signal. A signal computed on a close cannot be filled at that close. The fix is to enter on the next available price, so every trade uses only information that was actually available when the order was placed.
What is survivorship bias and why does it inflate backtest returns?
A survivorship-biased backtest includes only names that still trade today, removing companies that went bankrupt, merged, or were delisted. Since those failures were tradeable at the time, their losses never appear in the simulation. A universe of today's survivors inflates returns because the failures were removed before they could lose money.
Why should I plot the equity curve before and after costs?
If the curve's shape inverts after applying realistic commissions, spread, and slippage, the strategy was trading the cost model, not the market. A short-horizon strategy that flips from profitable to unprofitable once costs are applied has no live edge.
What makes a backtest reproducible?
Pin every random seed so the run is byte-reproducible. Record the data vintage and the commit hash for every reported result so the number can be traced back to its inputs. A result you cannot regenerate from a clean checkout is a result you do not actually have.
Sources
- The Probability of Backtest Overfitting , Bailey, Borwein, Lopez de Prado, Zhu, Journal of Computational Finance (2017)
- Advances in Financial Machine Learning , Marcos Lopez de Prado, Wiley (2018)
Keep the topic connected
- Checklist Trading Strategy Validation Checklist
A sign-off checklist for validating a trading strategy before risking capital: data hygiene, out-of-sample testing, trial accounting, deflated Sharpe, and risk backtests.
- Guide How to Validate a Trading Strategy
A step-by-step process for validating a trading strategy without fooling yourself: out-of-sample testing, trial accounting, deflated Sharpe, and a VaR backtest.
- Learn Look-Ahead Bias
Look-ahead bias: when a backtest accidentally uses data the strategy wouldn't have had at decision time. The most common variants and how to catch them.
- Learn Survivorship Bias
Survivorship bias in backtests: why dropped tickers, delisted funds, and dead share classes systematically inflate historical returns.