Nested model selection

Tuning on the reported folds is self-grading

Cross-validation is often described as if it produced one number independently of what happened before the number was calculated. It does not. If a study tries many feature sets, penalties, thresholds, or model configurations and reports the best score from the same folds used for the search, the folds have served two roles: they selected the procedure and they evaluated it. The reported score is then a score for a selected winner, not an untouched estimate of the search procedure.

The problem is present even when every candidate is individually evaluated with care. A sufficiently large search can find a configuration that looks good on the particular folds partly because of sampling variation. Choosing that configuration conditions the reported number on the search result. The issue is not that the winning model is necessarily useless; it is that the number has not been obtained from data kept outside the choice that produced it.

The inner and outer questions

Nested cross-validation separates the two roles. The inner loop uses only the outer-training partition to compare candidate configurations and select features, hyperparameters, or other data-dependent choices. The selected procedure is then refitted on the complete outer-training partition. The outer loop scores that fitted procedure once on its held-out partition, which was not available to the inner search. Repeating this over outer folds produces a collection of scores for the complete selection-and-fitting procedure.

The figure shows the order of operations, not a universal choice of fold count or candidate grid. The outer held-out partition remains outside the selection world. The estimate describes the declared grouped resampling and selection-and-fitting procedure; a model later retrained on all available data has different fitted weights.

The same boundary applies to more than a penalty parameter. Feature selection, scaling, imputation, harmonisation, and any other data-dependent input transformation must be fitted in the permitted training portion. When the intended estimand requires grouping, patient or cluster grouping must be respected by both loops. Otherwise a nested-looking diagram can still contain leakage through the data supplied to one of its steps.

Nested cross-validation estimates expected performance for the procedure that was repeatedly selected and fitted under the declared design. It does not by itself demonstrate performance at another site, establish a causal effect, or show that the chosen features represent the intended biology. Those are separate claims requiring their own designs.

A numerical sketch, not a measured result

Suppose a constructed example compares several lasso penalties using flat five-fold cross-validation and reports the winning AUROC as 0.88. When the same search is placed inside an outer loop, the mean outer score in the illustration might be 0.82. The difference is a schematic representation of selection optimism: the inner search was allowed to choose a winner, while the outer folds evaluate the procedure without participating in that choice. The values are not a result from a patient cohort and should not be carried into a study as an expected gap.

The useful comparison is not “nested is always lower”. It is whether the reported score belongs to a procedure whose choices were made without reading the observations used for that score. A prespecified model with no data-dependent selection may need a simpler design. A broad radiomics search with many correlated features and a small number of patients needs a design that exposes the search to genuinely held-out evaluations.

When the extra loop earns its cost

The computational cost of nested validation grows with the number of outer and inner fits. That cost is justified when selection is a substantial part of the analysis: many candidate features, aggressive tuning, multiple preprocessing options, or a small cohort in which chance differences can decide the winner. If the procedure was fixed in advance and the evaluation only estimates its performance, the selection problem is smaller, although the split, fitted-state, and uncertainty boundaries still apply.

A shorthand such as K x L for the cost is only schematic. Here K denotes the number of outer folds, and L must be understood as the complete inner workload per outer fold, including candidate fits, data-dependent preprocessing, and the refit on the outer-training partition. It is not a universal runtime formula. Grouped splitting is a separate design condition chosen to match the estimand and dependence structure, not a property supplied by nesting.

The final model used after development may be retrained on all permitted data, using a configuration selected according to the declared procedure. That model is not the object whose exact weights were evaluated in each outer fold. The outer scores describe expected behavior of the procedure under the resampling design, not a guarantee for one future fit or for an untested domain.

A bounded project connection

A project protocol can make this structure auditable by recording the split unit, the operations allowed to fit inside each loop, and the candidate-selection boundary before execution. A reproducibility check for the feature-generation process can make the inputs stable enough to compare, but it does not itself prove that nested validation was performed or that the resulting model generalises. Those conclusions remain properties of the recorded validation design and its evidence.

The next section explains why feature selection is especially fragile when the number of candidate columns is large relative to the number of patients.

Sources

• James G, Witten D, Hastie T, Tibshirani R, Taylor J. An Introduction to Statistical Learning with Applications in Python. Springer; 2023.
• Varma S, Simon R. Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics. 2006;7:91. doi:10.1186/1471-2105-7-91.
• Cawley GC, Talbot NLC. On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation. Journal of Machine Learning Research. 2010;11:2079-2107. JMLR paper.