Skip to main content
AI Fin Hub
Menu

Checklist, in Backtesting & Validation

Trading Strategy Validation Checklist

This checklist is the sign-off list for the strategy-validation process. Walk it before allocating capital and again whenever the strategy is modified.

12 items. Published May 26, 2026. By AI Fin Hub Research.

Your progress

Move item by item and keep your place

Progress saves locally, so you can work through the page over multiple sessions without resetting your checklist.

0/12 complete

Sections

Work in focused batches instead of one long wall

Section 1

Phase 1: Data hygiene

3 items

Use the Execution Simulator: Estimate execution cost in closed form — square-root permanent impact, linear temporary impact, half-spread, and a latency-drift band. See the slippage a naive backtest hides.

Section 2

Phase 2: Out-of-sample discipline

3 items

Use the Walk-Forward Validator: Upload a returns CSV. Rolling or expanding IS/OOS windows, per-window Sharpe, walk-forward efficiency, and a concatenated OOS equity curve. Catches regime.

Use the Walk-Forward Validation Visualizer: Paste a strategy returns CSV, get per-window in-sample vs out-of-sample Sharpe and the IS→OOS drop. Rolling and anchored window modes. Browser-only.

Section 3

Phase 3: Selection-bias correction

3 items

Use the Backtest Overfitting Score: Upload a backtest trade log and compute Probability of Backtest Overfitting (PBO), Deflated Sharpe Ratio, and the odds your edge survives live trading.

Use the Deflated Sharpe Ratio Calculator: Bailey & López de Prado deflated Sharpe — corrects observed Sharpe for selection bias across K trials. Reports deflated Sharpe, PSR (probability of skill).

Section 4

Phase 4: Risk and capacity

3 items

Use the VaR Backtest — Kupiec & Christoffersen: Paste P&L + VaR series and run Kupiec POF, Christoffersen independence, and joint conditional-coverage tests. Likelihood-ratio χ² p-values.

Use the Statistical Arbitrage Capacity Calculator: Maximum strategy AUM from alpha, slippage, fees, daily volume and volatility, and signal half-life. Square-root impact closed-form.

Use the Drawdown-Recovery Markov Simulator: Time to recover from an N% drawdown given monthly Sharpe + skew + kurtosis. Cornish-Fisher Monte Carlo, percentile distribution of recovery months.

Tips

Small moves that make the checklist easier to finish

Treat the holdout window as a one-shot exam. The moment you change the strategy in response to it, you have spent it, and the only honest fix is a fresh untouched window.
When the deflated Sharpe is marginal, gather more data or cut the number of trials rather than searching harder for a higher raw Sharpe. Searching harder is what created the problem.
Independence failures in the VaR backtest matter more than frequency failures. A model that is right on average but wrong in clusters will be wrong when you can least afford it.

Sources

  1. The Probability of Backtest Overfitting , Bailey, Borwein, Lopez de Prado, Zhu, Journal of Computational Finance (2017)
  2. Evaluating Interval Forecasts , Peter F. Christoffersen, International Economic Review (1998)

Keep the topic connected

  • Guide How to Validate a Trading Strategy

    A step-by-step process for validating a trading strategy without fooling yourself: out-of-sample testing, trial accounting, deflated Sharpe, and a VaR backtest.

  • Learn Overfitting

    Overfitting in trading-strategy backtests: how multiple-testing inflates apparent edges and the diagnostics that catch it.

  • Learn Walk-Forward Optimization

    Walk-forward optimization: rolling-window train/test that mimics live deployment. Why anchored vs sliding matters and the gotchas in window sizing.

  • Formula Deflated Sharpe Ratio Formula

    The deflated Sharpe ratio formula (Bailey and López de Prado, 2014): expected maximum Sharpe of N trials, skew and kurtosis terms, and the paper's example.