Medical imaging AI — reference

Reference · The task-family map and verified reading list. Teaching versions live in the chapter.

The task map

Task	What it produces	Key risk
Classification	a label per study/series/lesion	spectrum bias, miscalibration
Detection	bounding boxes / localisation of findings	false positives, small-object misses
Segmentation	a voxel mask per structure/lesion	overlap metrics hide boundary errors; reader variability
Registration	a spatial transform aligning two images	silently wrong alignment; FoR misuse (Ch. 4)
Reconstruction	an image from raw/sparse data	changes quantitative values downstream
Denoising / enhancement	a cleaner/sharper image	inventing structure; losing pathology
Harmonisation	images/features normalised across sites/scanners	removing real signal; leakage
Image synthesis	a synthetic image of an unacquired contrast/sequence/dose	hallucinating absent information
Quantitative measurement	a number (volume, diameter, attenuation)	geometry/segmentation-dependent
Prognosis / prediction	a risk/forecast	leakage, overfitting, calibration
Report generation	free-text / structured findings	fabricated findings; grounding
Multimodal (image+text+clinical)	fused prediction/report	leaking outcome via clinical fields
Workflow / agent orchestration	coordinated multi-step pipeline	compounding errors, no provenance

Most real systems combine several. The radiomics path is segmentation → measurement → phenotype → prediction; the foundation-model direction is one model, many tasks.

Segmentation metrics

• Dice / IoU — overlap; size-dependent.
• Hausdorff / 95%-Hausdorff / mean surface distance — surface error; complement overlap.
• Judge by downstream measurement, not one overlap number; report reader spread.

Tools / models

• nnU-Net (Isensee et al., Nat Methods 2021;18(2):203-211, PMID 33288961) — self-configuring baseline; ESTABLISHED EVIDENCE as the strong default.
• TotalSegmentator (Wasserthal et al., Radiol AI 2023;5(5):e230024, PMID 37795137) — ~104+ structures in one pass; MRI variant PMID 39964271.
• MONAI — PyTorch framework + tutorials for medical-imaging DL; integrates with 3D Slicer (MONAIAuto3DSeg).

Reconstruction

FBP → IR → DL. IR alters texture; DL can alter/invent texture. Record the method; do not pool across regimes; test under deployment reconstruction.

Synthetic imaging — verified reading

(literature map “Synthetic imaging” section):

• Spadea et al., Med Phys 2021, PMID 34407209 (MR→CT).
• Sim et al., Cureus 2025;17(10):e94561, PMID 41246632 (shoulder MRI DL-recon).
• Park et al., Eur Radiol 2025;35(12):7867-7876, PMID 40483292 (lumbar CT denoising).
• Herrmann et al., Diagnostics 2023;13(17):2747, PMID 37685285 (elbow MRI DL-recon).
• Choi et al., Sci Rep 2021;11:20403, doi:10.1038/s41598-021-00058-3 (synthetic contrast-enhanced CT from non-contrast CT; observer study; internal-only — the strong-evaluation pattern box in Synthetic Imaging and Invented Information).
• “RaD”, arXiv:2412.01496 (perceptual realism ≠ task utility).

Representation comparison (handcrafted vs learned) — verified

• Controlled benchmark (preprint) — arXiv:2607.01001 — foundation models vs 2D/3D radiomics, 7 classifier heads, 3 segmentation regimes, 5 tasks, LUNG1→LUNG2, worst-case cross-cohort metric: task-dependent dominance; radiomics competitive for survival; explicit label-derivation artifact for volume/stage.
• VPI fusion, external reversal — Insights Imaging 2026, doi:10.1186/s13244-026-02351-z — combined radiomics+ViT best in training only; externally 0.844 (radiomics) > 0.823 (combined) > 0.816 (ViT); combined wins on sensitivity, not AUC.
• RTOG 0617 — Br J Radiol Open 2023, doi:10.1093/bjro/tzae038 — foundational-AI vs radiomics vs clinical ≈ equal AUCs; three-way ensemble no gain (redundancy is a finding).
• Region × representation interaction — Radiat Oncol 2026, doi:10.1186/s13014-026-02896-9 — which representation wins depends on the ROI; hybrid best internally, 0.828 external.

Reading rule: the baseline ladder (clinical → volume → radiomics → learned → combinations) on identical splits; the external row decides; small internal fusion gains are variance until proven otherwise. Full treatment: Handcrafted and Learned Representations.

Foundation / generalist models — verified

• TotalSegmentator (PMID 37795137) — infra layer.
• Merlin — Nature 2026, doi:10.1038/s41586-026-10181-8, PMID 41781626 (preprint PMID 38978576). CT 3D VLM; large external validation.
• a2z-1 — arXiv:2412.12629 (preprint); generalist abdomen-pelvis CT.
• OmniMRI — arXiv:2508.17524 (preprint, 2025-08-24); generalist MRI VLM.
• Paschali et al., Radiology 2025;314(2):e240597, PMID 39903075 — read first for landscape/limits.
• ReCo — medRxiv 2026.07.14.26358025 (preprint); agentic/MCP orchestration example.

Progression: task-specific → pretrained → foundation → VLM → generalist → agentic. Ask of any paper: data scope, zero-shot vs adapted, external testing, licence, domain-shift limits. Most are preprints — read for evaluation methodology, treat “generalist” as orientation.

Evaluation, subgroup reasoning and trustworthy-AI guidance — verified

• FUTURE-AI — Lekadir K, et al. “FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare.” BMJ 2025;388:e081554. DOI 10.1136/bmj-2024-081554, PMID 39909534 (open access, PMC11795397; erratum BMJ 2025;388:r340). Six principles — fairness, universality, traceability, usability, robustness, explainability — operationalised as 30 lifecycle recommendations; expected compliance is stricter for deployable than proof-of-concept tools (full text inspected 2026-08-29).
• Subgroup differences in external evaluation are a signal to investigate among competing explanations (domain shift, case mix, prevalence, reference-standard differences, biology, sample size, confounding, harmful bias) — Evaluation and Subgroup Diagnosis works the reasoning; no universal fairness thresholds exist.

What usually goes wrong

• Ignoring anisotropic spacing in architecture/augmentation; augmentation that breaks HU/physical meaning.
• Reporting Dice without surface metrics or downstream-task impact.
• Pooling reconstruction methods; claiming generalisation from in-house data only.
• Treating a texture feature as a biomarker; treating a realistic synthetic image as “correct”; trusting a VLM’s generated finding without grounding.
• Deploying an agent/MCP pipeline with no provenance or audit (Ch. 10).

TRACE-CT connection

• TotalSegmentator-class segmentation → measurement is the spine TRACE-CT’s radiomics pipeline assumes (the SEG objects it decodes).
• Reconstruction as a variable is the reconstruction-changes-features effect, made real (Ch. 5).
• Foundation/generalist and synthetic-imaging items stay as reading; nothing in the current bounded experiment changes.

What to retain (chapter summary)

1. Medical images are spatial, anisotropic, scarce, and acquisition-dependent; geometry is part of the input.
2. Segmentation: judge by downstream measurement, not one overlap number; reader variability contextualises the reference standard (not a hard ceiling); nnU-Net/TotalSegmentator/MONAI are the tools.
3. Reconstruction (FBP→IR→DL) changes quantitative values — record it.
4. Synthetic imaging: “what is present vs what must be invented”; evaluate beyond pixel realism; hallucination/invented-information is the defining risk.
5. Foundation/generalist progression: task-specific → pretrained → foundation → VLM → generalist → agentic; read for data scope, external testing, and domain shift.
6. Representation comparison: handcrafted, learned and foundation embeddings are complementary candidates, not rivals by default — demand the baseline ladder, identical splits, and the external row before believing a fusion gain.