Skip to main content
AI Fin Hub
Menu

Checklist, in Backtesting & Validation

Overfitting Red-Flags Checklist

Overfitting is selection bias wearing a backtest. The more configurations you try, the higher the best-looking Sharpe rises by luck alone. This checklist is a list of warning signs to scan for before allocating capital and after any round of optimization. Essential flags are near-certain disqualifiers if unaddressed; recommended flags warrant deeper investigation; nice-to-have flags help you quantify how much of the result is real.

12 items. Published May 26, 2026. By AI Fin Hub Research.

Your progress

Move item by item and keep your place

Progress saves locally, so you can work through the page over multiple sessions without resetting your checklist.

0/12 complete

Sections

Work in focused batches instead of one long wall

Section 1

Phase 1: Search-process flags

3 items

Use the Backtest Overfitting Score: Upload a backtest trade log and compute Probability of Backtest Overfitting (PBO), Deflated Sharpe Ratio, and the odds your edge survives live trading.

Use the Deflated Sharpe Ratio Calculator: Bailey & López de Prado deflated Sharpe — corrects observed Sharpe for selection bias across K trials. Reports deflated Sharpe, PSR (probability of skill).

Section 2

Phase 2: Parameter fragility flags

3 items

Section 3

Phase 3: Equity-curve flags

3 items

Section 4

Phase 4: Generalization flags

3 items

Use the Walk-Forward Validator: Upload a returns CSV. Rolling or expanding IS/OOS windows, per-window Sharpe, walk-forward efficiency, and a concatenated OOS equity curve. Catches regime.

Tips

Small moves that make the checklist easier to finish

The trial count is the single number that turns a raw Sharpe into an honest one. If you cannot state it, you cannot claim the strategy is validated.
Robust edges are plateaus, fragile ones are spikes. When you optimize, look at the whole parameter surface, not just the coordinate of the maximum.
When the deflated Sharpe is marginal, the cure is fewer trials or more data, never a harder search. Searching harder is exactly what manufactured the overfit.

Run the numbers next

  • Playgrounds Walk-Forward Validation Visualizer

    Paste a strategy returns CSV, get per-window in-sample vs out-of-sample Sharpe and the IS→OOS drop. Rolling and anchored window modes. Browser-only.

Sources

  1. The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality , Bailey, Lopez de Prado, Journal of Portfolio Management (2014)
  2. The Probability of Backtest Overfitting , Bailey, Borwein, Lopez de Prado, Zhu, Journal of Computational Finance (2017)

Keep the topic connected

  • Checklist Trading Strategy Validation Checklist

    A sign-off checklist for validating a trading strategy before risking capital: data hygiene, out-of-sample testing, trial accounting, deflated Sharpe, and risk backtests.

  • Learn Overfitting

    Overfitting in trading-strategy backtests: how multiple-testing inflates apparent edges and the diagnostics that catch it.

  • Formula Deflated Sharpe Ratio Formula

    The deflated Sharpe ratio formula (Bailey and López de Prado, 2014): expected maximum Sharpe of N trials, skew and kurtosis terms, and the paper's example.

  • Learn Probability of Backtest Overfitting (PBO): Bailey et al.

    Probability of Backtest Overfitting (PBO): Bailey, Borwein, López de Prado and Zhu, Journal of Computational Finance 20(4), 2017, SSRN 2326253, via CSCV.