The measurement layer

The preceding section described how a feature depends on its immediate inputs. The wider question is where those inputs sit between the patient and a clinical claim. A useful path is biological reality -> acquisition -> reconstruction -> region/segmentation -> geometry/preprocessing -> feature implementation -> measurement -> inference -> model -> claim or decision.

Each arrow represents a transformation or an interpretive step, not a promise that the next box preserves the meaning of the previous one. A reconstruction can change image texture. A segmentation can change the tissue included in the calculation. Resampling and discretisation can change spatial and intensity relationships. The implementation turns those prepared data into a defined number. The number is therefore a measurement artifact of a stated pipeline, not a value that belongs to the tumour independently of how it was observed.

Why the layer boundary matters

A model score cannot repair an unspecified or unsuitable measurement layer. If feature values encode reconstruction, site, scanner, segmentation style, or preprocessing, a downstream model may learn those signals. They can be systematic and strongly associated with the outcome in one dataset, even when they do not represent the biological mechanism the study intends to measure. A high AUROC or C-index can therefore coexist with poor measurement validity and poor performance after the input mix changes.

The possible shortcuts are concrete. A sharp and a smooth reconstruction can produce different texture patterns. A site can be associated with both a scanner protocol and a patient mix. A manual and a model-generated contour can include different boundary voxels. A second resampling pass or a different binning rule can alter the values before extraction. Two software versions can expose the same feature label while resolving different defaults or aggregation conventions. None of these possibilities is diagnosed by looking only at the final score.

The important qualification is that a scanner or site signal is not automatically non-biological. Site may encode a real population difference, a care pathway, or a technical nuisance, and these explanations can coexist. The measurement question is whether the intended estimand has been isolated well enough that the claim does not silently rely on an uncontrolled proxy.

Name the failure correctly

Three terms are often used as if they described the same problem. They do not. The distinction matters because the repairs are different:

Failure mode	What has happened	Typical consequence	Main response
Data leakage	Information unavailable at evaluation has entered training or model selection	The reported evaluation is optimistic because the held-out boundary was breached	Keep fitted and selected steps inside the validation loop and prevent subject overlap
Confounding or shortcut learning	A nuisance variable legitimately present in the input is associated with the outcome and is used as a proxy	The model can predict in the study while relying on a signal other than the intended biology	Improve design, stratification, measurement characterization, adjustment, and external testing
Domain shift	The input distribution or the relationship relevant to deployment differs from development	A shortcut or calibration learned in one setting may not transport	Test the relevant shifts and characterize transportability, with adaptation declared when used

Fitting a transformation using the full dataset before cross-validation is leakage when the held-out data influence the fitted state or selection. By contrast, a kernel fingerprint already present in every feature value is not leakage merely because the model uses it. If that fingerprint is correlated with severity, it is a candidate shortcut or confound: its causal role and transportability must be investigated rather than assumed from the correlation alone. If the scanner mix later changes, the same problem appears as domain shift. The labels identify different locations of failure; they should not be exchanged as general warnings.

Four questions that the chain keeps separate

The chain separates four questions that are often compressed into one performance number. First, measurement validity and stability: does the number represent the intended target, and how much does it move when the target or the acquisition is supposed to be unchanged? Second, association: is the number related to another variable in the study? Third, prediction: does a model using it discriminate or calibrate for the stated task? Fourth, utility: does using the result improve a decision or outcome? A feature can be associated with an endpoint without being a stable measurement of the intended biology. A model can predict with a shortcut without establishing that biology. A valid prediction can still have no useful decision consequence.

Agreement and robustness analysis address the measurement layer. Statistical association and predictive metrics address later questions. Reader studies can assess diagnostic performance and may assess aspects of clinical use, while decision or impact analyses provide stronger evidence about decision or outcome consequences. Keeping the questions separate prevents a result at one layer from being presented as evidence for a conclusion at another.

A bounded measurement-only application

The logic becomes tangible in a test-retest or reconstruction-robustness study. Use paired scans from the same subjects, hold the region, geometry, preprocessing, and feature implementation fixed, and deliberately vary the reconstruction axis. Calculate agreement and its uncertainty, then describe what changed. Do not add a survival endpoint, harmonisation step, or feature-selection claim to the same experiment unless those are separate, prespecified questions.

The teaching value of this design is the experimental separation: a poor repeatability result is not a failed study when the question is whether the measurement survives a declared perturbation. It is evidence about the measurement layer, and it tells a later modelling study what cannot safely be treated as stable. A reader can understand that design without access to project files, hashes, or chronology; exact project identity and provenance belong in companion material rather than serving as prerequisites for the argument.

The next section makes the implementation part of that contract. A feature name is a useful handle, but it is not yet an operational definition.

Sources

• Zwanenburg A, Vallières M, Abdalah MA, et al. The Image Biomarker Standardisation Initiative. Radiology. 2020;295(2):328-338. doi:10.1148/radiol.2020191145, PMID 32154773.
• Kocak B, et al. CLEAR radiomics reporting checklist. Insights into Imaging. 2023;14:75. PMID 37142815, doi:10.1186/s13244-023-01415-8.
• Quantitative Imaging Biomarkers Alliance (QIBA), RSNA — official QIBA information.
• Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement. BMJ. 2024;385:e078378. doi:10.1136/bmj-2023-078378.
• Bernett J, et al. Guiding questions to avoid data leakage in biological machine learning applications. Nature Methods. 2024;21:1444-1453. doi:10.1038/s41592-024-02362-y.