Skip to content
Imaging Evidence Path

Chapter 6 — Medical imaging AI

This chapter · ~3 h total across 8 lessons · The domain map of medical-imaging AI · Energy: low to medium · Setup: optional Slicer/MONAI for the segmentation lessons

This is not “what is a neural network” — you know that. This chapter is the domain map: what changes when the inputs are medical images, how segmentation is measured, how reconstruction feeds quantification, where synthetic imaging and foundation models sit, and the failure modes that recur across all of it.

The thread this chapter pulls

Medical-imaging AI is not general vision with bigger tensors. The inputs carry physical geometry, scarce and expensive labels, and acquisition-dependent distributions; the outputs feed measurements that must be trustworthy. This chapter builds the intuition for what is different about imaging AI, then walks the specifics: why segmentation is infrastructure, why Dice alone is a bad verdict, why nnU-Net and TotalSegmentator became defaults, how reconstruction can change the biology you measure, what synthetic imaging is really being asked to invent, and where foundation/multimodal models genuinely change the picture (and where they do not).

Lessons

  1. Why medical images are not just large tensors — geometry, spacing, acquisition, labels, domain shift.
  2. Segmentation as infrastructure — why so many downstream tasks depend on masks.
  3. Why Dice is not enough — size dependence, surface errors, reader variability, downstream measurement.
  4. Why nnU-Net became such a strong baseline — self-configuration, engineering, TotalSegmentator.
  5. Reconstruction can change the biology you measure — FBP/IR/DL and quantitative consequences.
  6. Synthetic imaging: what is the model inventing? — denoising vs missing contrast; hallucination risk.
  7. Foundation and multimodal models — what is new, what is not, evidence boundaries.
  8. Evaluating an imaging model beyond one metric — task-specific evaluation, external testing, calibration, failure modes.

The task-family map (classification, detection, registration, synthesis, …) and the verified reading list are in the imaging-AI reference.

How this chapter connects

  • It assumes Chapters 1–2 (images carry geometry and acquisition metadata) and Chapter 3 (measurements depend on masks and pipelines).
  • Chapter 7 applies the evaluation lens to papers; Chapter 9 takes a validated model to deployment.

If you only have an hour, lessons 3 (Dice is not enough), 5 (reconstruction) and 6 (synthetic imaging) carry the ideas most papers get wrong.