Chapter 11 — Clinical Deployment, Human-AI Systems, and Lifecycle

A model can produce a convincing result on a held-out dataset and still be the wrong thing to place in a clinical workflow. The result may not reach the person who needs it, may be presented at the wrong time, or may change how a reader behaves. A score that was useful in the development population may create an unmanageable alert load at another site. A model can also remain technically unchanged while scanners, protocols, populations, clinical definitions, or software around it change. Clinical deployment is therefore not the last box after validation. It is a lifecycle in which the claim, the system, the people, and the evidence have to remain aligned.

A validated model is not yet a clinical system

The narrow question answered by a held-out test set is whether a specified model discriminates or predicts on a specified evaluation population under the conditions of that evaluation. Even a strong AUROC does not by itself establish that adding the model to a radiologist improves reading, that the result can be retrieved and understood in a particular workflow, or that an operating point is appropriate for a new population. It says still less about how readers respond to wrong suggestions or whether the system remains reliable after the surrounding world changes.

The word validated must therefore be completed with a claim. A frozen external test can support a transportability claim of a particular scope. It does not silently become evidence of workflow benefit, patient benefit, safety, efficiency, or clinical utility. Those are different estimands and require different observations. The distinction is the same one used throughout Chapter 10: an object can be technically correct while its identity, timing, retrieval path, rendering, provenance, or custody is wrong for the intended use.

The deployment gap is best understood as a set of questions that interact without collapsing into one another. What role is the AI intended to play? Is the relevant comparison a model against a reference, a reader alone against a reader with AI, or a workflow before and after integration? Which threshold turns a score into an action, and what prevalence, workload, and error consequences surround that action? Can a reader notice and override a wrong suggestion? What can be observed before outputs affect care, and what will be monitored afterward? Who owns updates, incidents, rollback, and the decision to continue? No test-set metric answers this whole chain.

From model result to lifecycle decision

The chapter follows that chain in the order a deployment decision often has to be made. It first fixes the intended role, then separates standalone model performance from reader-plus-AI evidence. It uses an explicitly bounded reader-study example to show how evidence scope is preserved, and then treats thresholds as operating-point decisions rather than portable properties of a model. The next step is the human-plus-model system: automation bias, alert load, cognitive offloading, and explanations can change the effect being measured. Staged and shadow evaluation then connects retrospective evidence to local workflow, while monitoring, versioning, and rollback keep validation open after go-live. The final sections connect this lifecycle to intended purpose, research-to-product boundaries, governance, and adoption.

The sequence is a reasoning spine, not a universal deployment recipe. Local governance, the clinical role, the risk of an error, the available reference data, and the reversibility of a change determine which evaluations are appropriate and in what order. The durable rule is to state what each stage is meant to learn and not use a result from one stage as a substitute for evidence from another.

The boundary of this chapter

The discussion assumes the validation concepts in Chapter 7, the imaging workflow and result-transport concepts in Chapter 10, and the evidence-to-claim discipline in Chapter 9. It applies those ideas to the move from a model result to a human and organizational system. The research-to-product discussion later in the chapter is an EU-scoped, illustrative orientation, not a regulatory-compliance checklist or legal advice. It does not classify any real product, and no DICOM transformation can determine a legal classification or intended purpose.

The first deployment decision is consequently not which metric to report. It is what the system is supposed to do, for whom, and at which point in the work. That role defines the evidence that follows.

The reporting and appraisal distinctions used here are oriented by TRIPOD+AI, PROBAST+AI, and CLAIM 2024. These are reporting and appraisal guides, not certificates of clinical benefit.