Evaluation and Subgroup Diagnosis

A headline metric is a partial observation

Every model class has a metric that is easy to headline. Dice summarises segmentation overlap, AUROC summarises one aspect of classification discrimination, and SSIM or PSNR summarises one kind of image similarity. Each can be useful. None is a verdict on a model’s complete purpose. The missing facets are determined by the task: spatial error and downstream measurement for a mask, calibration and decision consequences for a risk score, or pathology and task utility for a synthetic image.

Uncertainty also needs a precise object. An interval around an aggregate AUROC describes uncertainty in an estimated population summary under a stated estimand and interval or resampling procedure. The report should name both. It is not automatically an interval for an individual patient’s future outcome or risk. Calibration uncertainty, uncertainty in a model’s performance, and uncertainty in an individual prediction can arise from different sources and should not be collapsed into the phrase “patient-level confidence interval” without defining what is being estimated.

Evaluation follows the task and the claim

For segmentation, overlap should be read with surface behaviour, structure- specific results, reference-reader variability, and the downstream measurement the mask enables. For classification, discrimination should be separated from calibration, predictive values, threshold-dependent utility, and uncertainty. For synthesis, visual or pixel similarity should be accompanied by anatomical, pathology, quantitative, and downstream-task checks. Detection, registration, and other tasks likewise need measures that correspond to their output and use; there is no model-independent checklist item that can replace the task contract.

The evaluation setting is part of the claim. Internal, temporal, and external tests expose different changes in population, acquisition, site, or time. The report should state whether the model and its threshold were frozen, recalibrated, adapted, or retrained for the evaluated setting. A result after adaptation may be valuable, but it is not the same result as a frozen-model transport test.

Failure analysis makes the average interpretable. Per-structure, per-subgroup, and case-level inspection can reveal errors concentrated in small structures, unusual anatomy, a shifted acquisition, or a particular operating point. The subgroups and analyses should be declared before results are repeatedly searched, and estimates should carry uncertainty appropriate to their unit and metric. AUROC is conventionally unchanged by class prevalence only when the class-conditional score distributions are fixed, while predictive values and threshold-dependent utility do depend on prevalence and operating policy. A reported performance difference must first be checked for metric and input comparability before it is explained as a property of a site or population. A site comparison therefore cannot be interpreted through prevalence alone: spectrum, inputs, labels, and score distributions must be checked first.

A subgroup gap opens a hypothesis space

Consider the pinned synthetic site comparison in the figure. It is deliberately constructed to show an overall score alongside an outlying site, not to report a hospital, scanner, or clinical result. The first question is mechanical and precedes the causal hypotheses: did the sites use the same metric, score definition, threshold policy, reference procedure, and comparable inputs? A difference caused by a changed operating point or metric is not evidence that the model behaves differently in the population represented by the site.

Synthetic point estimates, not hospital or model results; no uncertainty interval is shown. Confirm the metric, score, threshold policy, reference procedure, and input contract before entering H1-H7. The seven equal boxes are non-ranked, non-exhaustive hypotheses to investigate, not findings; a gap alone proves neither a cause nor harmful bias, and more than one mechanism may coexist.

With comparability established, keep the following seven explanations separate. Treat them as H1-H7: explicit, non-ranked, non-exhaustive hypotheses to investigate, not findings; more than one may coexist.

1. H1 — Technical or domain shift: scanner, reconstruction, dose, protocol, or site conditions may move the input distribution. The response is to measure the shift and test an appropriate adaptation or training strategy, not to call the gap biological by default.
2. H2 — Case mix or spectrum effect: the subgroup may contain a different mixture of disease severity, lesion size, treatment history, or referral context. The same model can perform differently on a different mixture without that alone establishing unfairness.
3. H3 — Reference-standard differences: labels or annotation procedures may differ between sites. The apparent gap may partly describe the target used for scoring rather than a change in the model’s behaviour.
4. H4 — Biology: disease may have different imaging expression in a subgroup. A performance difference can reflect real variation, but that interpretation still requires evidence and does not automatically license a deployment claim.
5. H5 — Small-sample uncertainty: a subgroup estimate can be much less precise than the aggregate. Intervals or another suitable uncertainty analysis are needed before a point estimate is treated as a stable gap.
6. H6 — Hidden confounding: the subgroup variable may proxy an unmeasured factor, such as referral pathway, comorbidity, prior imaging, or acquisition era. The proxy and its relationship to the outcome need investigation.
7. H7 — Genuine harmful bias: harm to patients in a subgroup is the defining concern. It is not a residual category that becomes possible only after shift, label differences, and confounding have been excluded; harmful effects may act through those mechanisms or coexist with them.

These are competing explanations for an observed signal, not a ranking and not an exhaustive taxonomy. Each points to a different next analysis. Technical shift may call for acquisition characterization and transport testing. Case mix may call for stratified reporting. A reference difference calls for annotation review. Small samples call for more informative uncertainty. A confound calls for design or measurement work. Evidence of harmful patient impact can change a deployment decision even when another mechanism also contributes.

The claim changes with the investigation

A bare aggregate score is incomplete once a meaningful subgroup gap has been observed. The responsible statement includes the subgroup result, its uncertainty, the metric and input contract, and the explanation that remains under investigation. It should not jump from a gap to a claim of unfairness, but neither should it dismiss the gap as harmless biological variation without checking the alternatives and the patient consequences.

Pre-specification matters because searching many subgroups and many metrics after seeing the results increases the chance of finding an apparent difference by chance. It does not make a discovered post-hoc signal irrelevant; it changes how strongly it can be presented and what confirmation is needed. The additional evaluation should be proportional to the claim: metric comparability, case-mix and label review, uncertainty, calibration, predictive values, threshold policy, and task utility may all be relevant, but they answer different questions.

This is the evaluation lens to carry into paper appraisal: reconstruct the task, the reference, the tested setting, the fitted state, and the failure modes before accepting a single score. The final section makes the same logic explicit for representation comparisons, where a more complex feature table must earn its increment over simpler clinical and measurement baselines.