Synthesis and the limits of inferred information

Ask what the source actually contains

The preceding sections show why an MR image is not simply a photograph with a stable brightness scale. It is the result of a particular preparation, timing, sampling, and reconstruction path. That fact gives a precise way to read an image-synthesis claim: before asking whether the output looks realistic, ask which target information was actually present in the source and which part must be inferred.

Consider four common task families. Denoising starts from a measurement of the target contrast whose signal has been degraded by noise or another known limitation. The model may still remove a small lesion or smooth a boundary, so the target is not automatically faithful, but the task is framed as recovering a degraded observation. T2-to-T1 or T2-to-STIR translation starts from the same broad anatomy but asks for a different contrast. Shared anatomy helps, yet a single source contrast does not uniquely determine the target contrast, so the output remains an estimate of an unacquired measurement.

MRI-to-synthetic-CT asks for information associated with x-ray attenuation or electron density that MRI does not directly measure. In applications such as MR-based radiation- therapy planning, a synthetic CT may be intended to provide an image suitable for an attenuation or dose-calculation workflow. The anatomy is observed through MRI, while the target physical quantity is inferred from MR-visible cues and the relationship learned from paired or otherwise suitable data.

Pre-contrast-to-post-contrast T1 is a different boundary again. The pre-contrast scan contains anatomy and baseline signal, but it does not contain the later distribution of the administered contrast agent. A model can learn associations that make a predicted post-contrast image plausible; it cannot turn that prediction into a measurement of the enhancement that was never acquired.

Task family	What is shared with the target	What remains inferred
Denoising	a measurement of the target contrast, with degradation	the clean signal and preservation of small findings
T2 to T1 or STIR	anatomy and related MR observations	the target sequence contrast and its unmeasured details
MRI to synthetic CT	anatomy and MR-visible cues	attenuation/electron-density-related information
Pre-contrast to post-contrast T1	anatomy and baseline signal	contrast-agent distribution and enhancement

The table is a way to state the information boundary, not a universal ranking of task risk. In every row, the output must be treated as a conditional estimate whose validity depends on the acquisition, training population, pairing, and reconstruction assumptions.

Realistic pixels do not settle the question

A synthetic image can be visually convincing while changing anatomy, obscuring pathology, altering a quantitative measurement, or supplying a downstream model with misleading features. Pixel-similarity or perceptual metrics can describe one aspect of the output, but they do not by themselves show that a lesion was preserved, that a measured property was recovered, or that a clinical task remains valid.

An evaluation should therefore ask several separate questions. Is the anatomy consistent with the source and with an independently available reference? Is pathology preserved, neither removed nor introduced? Do volumes, intensities, or radiomic measurements agree when the synthetic and acquired images are compared under a prespecified procedure? Does the downstream task or reader use the synthetic image without a loss that a pixel metric would hide? Does the method expose uncertainty or failure cases when the required target information is ambiguous? Finally, has the evaluation survived an acquisition or site change that was not used to tune the method?

These questions are especially important for pre-contrast-to-post-contrast synthesis. The absence of measured enhancement is not repaired by a sharp image, a low reconstruction error, or a confident model output. If the target contains information that the source cannot distinguish, the model may produce the most probable appearance for its training population rather than the patient’s unmeasured state. The proper claim is consequently about the performance and limits of an inference procedure, not about recovering a measurement that never occurred.

The chapter’s measurement boundary

The same reading habit applies to every synthetic-MRI paper. Reconstruct the source and target acquisitions, identify the information shared between them, and then separate image similarity from anatomical, pathological, quantitative, task, uncertainty, and transportability evidence. A method that succeeds on one of those questions has not automatically succeeded on the others.

MRI has thus brought the chapter back to the general measurement problem. Signal becomes contrast through a protocol, contrast becomes an image through sampling and reconstruction, and a synthetic target may add an inference layer on top. Before a downstream analyst can trust a value, the image must also be represented and stored with enough information to interpret its pixels. That is the subject of Chapter 3’s stored-values entry, “from stored values to physical meaning”.

Sources

• Spadea et al., “Deep learning based synthetic-CT generation in radiotherapy and PET: A review” — public review of MR-to-synthetic-CT problem framing and radiation-therapy/PET applications; also indexed at PMID 34407209.
• Smith TE, Steven A, Bagert BA. Gadolinium Deposition in Neurology Clinical Practice. Ochsner J. 2019;19:17-25. doi:10.31486/toj.18.0111. (Full text)
• Heckel R, Jacob M, Chaudhari A, Perlman O, Shimron E. Deep learning for accelerated and robust MRI reconstruction. Magn Reson Mater Phy. 2024;37:335-368. doi:10.1007/s10334-024-01173-8. (Full text)
• Bahloul MA, Jabeen S, Benoumhani S, et al. Advancements in synthetic CT generation from MRI: A review of techniques, and trends in radiation therapy planning. J Appl Clin Med Phys. 2024;25:e14499. doi:10.1002/acm2.14499. (Full text)
• Wang T, Lei Y, Fu Y, et al. A review on medical imaging synthesis using deep learning and its clinical applications. J Appl Clin Med Phys. 2021;22:11-36. doi:10.1002/acm2.13121. (Full text)
• Cohen JP, Luck M, Honari S. Distribution Matching Losses Can Hallucinate Features in Medical Image Translation. MICCAI 2018:529-536. doi:10.1007/978-3-030-00928-1_60. (Author manuscript)
• Chung M, Calabrese E, Mongan J, et al. Deep Learning to Simulate Contrast-enhanced Breast MRI of Invasive Breast Cancer. Radiology. 2023;306:e213199. doi:10.1148/radiol.213199. (Full text)