Time-based splitting alone is not enough, because crypto strategy return labels often span multiple periods, and adjacent data points overlap and are autocorrelated. When you split data into training and test sets by time, it looks like the training set comes before the test set, but the return outcomes of the samples at the end of the training set may only be fully revealed after the test set begins. The model "sees" information it should not have known during training, so backtest results become overly optimistic, while live performance does not match.
A leading global cryptocurrency platform,suitable for both beginners and experienced traders.
New user benefit: 20% off trading fees upon registration!！
Time-based splitting fixes "direction" but not "overlap"
The core contribution of time-based splitting, or walk-forward validation, is respecting the arrow of time: the training set only uses data before the test set, avoiding using the future to predict the past. This is far more correct than randomly shuffled K-Fold. Random K-Fold causes serious temporal leakage in time series. A model might use data from January 3 to "predict" January 2, but in live trading you can never use tomorrow's data to predict today.
However, time-based splitting only handles the directional problem of "training set before test set." It does not handle label overlap. Suppose your strategy predicts returns over the next 5 days. The last training sample is January 31, and its label is not fully determined until February 5. If the test set starts on February 1, then the label windows of the last few training samples overlap with the time window of the test set. The model has already "felt" the market movements during the first few days of the test period, because those movements also affected the labels of the training samples.
López de Prado explains this problem clearly in Advances in Financial Machine Learning: one way financial time series violate the IID assumption is through overlapping labels. If you predict 5-day forward returns, observations at time t and t+2 share 3 days of the same price movement. Time-based splitting does not automatically remove this shared information.
Feature autocorrelation spreads leakage from the "boundary" to the "neighborhood"
Even if you handle label overlap, the long memory of features creates a second layer of leakage. Common crypto strategy features such as moving averages, volatility estimates, and momentum indicators are not independent point values. They smooth historical windows, so feature values at adjacent time points are highly correlated.
This means that a small portion of training data right before the test set still has features highly similar to the test period. The patterns the model learns at the end of the training set may simply be an "early adaptation" to the market state at the beginning of the test period, rather than a truly generalizable signal. López de Prado's proposed solution is to add an embargo buffer after the test set, discarding a small portion of training data immediately after the test set ends, typically 1% to 5% of the total data length. Time-based splitting itself does not include this buffer.
When comparing multiple strategies, time-based splitting gives you only one path
Crypto strategy research rarely tests only one parameter combination. You adjust grid intervals, moving average periods, take-profit and stop-loss ratios, and may run dozens or hundreds of configurations. Time-based splitting cuts the data into a linear "past → present → future" path. You compare all configurations on this single path and pick the best performer. The problem is that you only see results under this one split path.
Combinatorial Purged Cross-Validation, or CPCV, takes a different approach: it cuts the data into multiple blocks, and the test set is no longer a single continuous segment but a combination drawn from different positions. This gives you multiple training and testing paths, allowing you to observe the performance distribution of a strategy when different market segments serve as the test set, rather than a single point estimate. If a strategy performs well on only one time-based split path and collapses on another, it means the strategy is fitted to that specific historical segment, not a repeatable mechanism.
Time-based splitting is still necessary, but not sufficient
Time-based splitting is the starting point of correct validation, not the finish line. For crypto strategies, you should add at least three things on top of it:
Set a purge window equal to your label period. If the strategy predicts returns over the next N days, the last N days of training samples must be removed from training, because their outcomes overlap with the test period.
Set an embargo buffer, typically 1% to 5% of the data length, discarding a small portion of training data after the test set to cut residual leakage caused by feature autocorrelation.
Record how many parameter sets you searched. If you compared 200 configurations and only report the backtest performance of the best one, even with purging and embargoing, that "best" result is still picked from noise. Methods such as Bailey and López de Prado's Deflated Sharpe Ratio are designed to correct this selection bias. A backtest without recording the number of searches cannot tell whether the result is a signal or survivorship bias.
A practical rule of thumb: if your strategy's out-of-sample performance goes from "great" to "average" or even "unusable" after adding purging and embargoing, it means a meaningful part of the previous good-looking result came from leakage. The strategy did not get worse. You are simply seeing numbers closer to live trading for the first time.
A leading global cryptocurrency platform,suitable for both beginners and experienced traders.
New user benefit: 20% off trading fees upon registration!！
References
- Hugging Face·Cross-Validation in Finance: Purging and Embargoing，2026-11-01
- PyPI·purgedcv: scikit-learn-compatible purged and combinatorial cross-validation，2026-11-01
- GitHub·Why Time Series Is Different，2026-11-01
- Parallax Research·Out-of-Sample Validation Protocols: Guarding Against Overfitting in Quantitative Screening，2026-11-01
- ScienceDirect·Backtest overfitting in the machine learning era: A comparison of out-of-sample testing methods，2026-11-01
