Skip to content
Imaging Evidence Path

Clinical deployment and lifecycle — reference

Reference · Dense lookup — deployment path, AIRA/AIW-I links, agentic-workflow risks, reading. Teaching versions live in the lessons.

Deployment roles

Autonomous / standalone · concurrent assist · second-reader · triage / prioritisation · pre-populated measurements / report suggestions. Role decides metric, threshold, and reader-study design. AI-alone ≠ reader+AI — the relevant number is the delta.

Reader studies

  • MRMC (multi-reader, multi-case) — standard design; ROC/AUROC per reader, averaged.
  • Crossover + washout — each reader reads each case both unaided and with AI, counterbalanced, separated by a washout to control memory/order effects.
  • Report sensitivity, specificity, AUROC, reading time, inter-reader agreement; probe automation bias (do AI errors become reader errors?).

A sensible prospective path

retrospective development
  → independent external testing (frozen model)
  → silent / shadow deployment (outputs hidden, compared to real reads)
  → reader / clinical-workflow study
  → prospective deployment (live, with oversight)
  → post-deployment monitoring

Deployed thresholds

Not the AUROC-optimal point. Depend on prevalence, workflow role (triage favours sensitivity; confirmation favours specificity), FP/FN cost, and calibration in the deployed population. Recalibrate per site; thresholds re-estimated on external data are adaptation — report them.

Human factors

Automation bias · alert fatigue · context blindness · cognitive offloading · failure to override wrong suggestions · miscalibrated trust. Measurable only if the evaluation includes the human (reader study, shadow deployment).

Drift taxonomy

Data · protocol/scanner · population · label/clinical-practice · performance. Monitoring needs continuous metrics, drift detectors, alerting, versioning/provenance, rollback. IHE AIRA is the standards-level support for assessment/feedback/monitoring. Model updates restart validation (scaled to the change); continuous learning blurs the train/test boundary.

Trust stack

standalone correctness → reader+AI delta → integration → operating point/calibration → human-factors safety → deployment staging → lifecycle monitoring. Absent layers are absent trust.

Agentic workflows (emerging architecture)

Agent / MCP-style orchestration (an LLM coordinating tools) is an emerging deployment shape. ReCo (Tzanis & Klontzas, medRxiv 2026.07.14.26358025, VERIFIED preprint; github.com/eltzanis/ReCo) integrates Orthanc + Merlin + TotalSegmentator + dosimetry via MCP servers as one example. Added risks: compounding errors, end-to-end provenance, scoped/auditable tool permissions, end-to-end (task-level) evaluation. The generalist-AI direction (e.g. Dogra et al., GRAI concept, Radiology 2025;316(3), doi:10.1148/radiol.242362, VERIFIED review/perspective) shares these lifecycle concerns.

Try it (practical exercises)

Pick one: (1) critique a published reader+AI study (e.g. Herpe et al.) against the MRMC-crossover/washout checklist; (2) draw the deployment lifecycle for a TotalSegmentator-class assistant, naming the human-factors risk and drift signals at each stage; (3) design the monitoring metrics (drift axes, provenance fields, thresholds, AIRA-style feedback) a TRACE-CT-like quantification tool would need if deployed.

Read / watch

What to retain (chapter summary)

  1. AI-alone performance ≠ reader+AI performance; measure the delta in a proper reader study (MRMC, crossover, washout).
  2. A sensible path runs retrospective → external → shadow → reader study → prospective → monitoring.
  3. Deployed thresholds depend on prevalence, role, and FP/FN cost — and need calibration.
  4. Human factors (automation bias, alert fatigue, trust calibration) decide whether a model helps or harms.
  5. Deployment is not the end: monitor data/protocol/population/label/performance drift; AIRA is the standards-level support.
  6. Model updates (and continuous learning) restart validation; agents add compounding-error and provenance risks.