Geometry, Modality, and Augmentation Constraints

A medical image is a located, acquired measurement

Calling an image a tensor is useful for implementation, but insufficient for reasoning. In a CT volume, for example, the array values are the result of a reconstruction and are interpreted together with a spatial grid and an acquisition contract. In a calibrated CT representation they may be related to attenuation on a Hounsfield scale, but that interpretation is conditional on the stored values and transformations being handled correctly. A natural RGB image does not carry the same intensity semantics, spatial sampling, or acquisition history.

Spacing changes the physical meaning of a network operation. A convolutional receptive field spanning the same number of voxels covers a different distance when the voxels are thick in one scan and fine in another. A five-millimetre through-plane spacing and a 0.7-millimetre spacing are not interchangeable just because both volumes can be represented by arrays with three indices. Resampling to a declared grid, or designing a spacing-aware model, can make the intended physical scale explicit. Neither choice makes the original sampling difference disappear; the grid and the operation used to change it remain part of the model contract.

Location is equally important. Origin, spacing, and direction connect an array index to physical space. A transformation applied to an array is not automatically the same as a transformation of the patient. Image, mask, and other spatial objects must be transformed under a compatible coordinate convention, or a visually plausible result can associate a prediction with the wrong anatomy. The geometry material in the voxel-to-patient discussion and the resampling discussion provides the longer treatment; here the consequence is that geometry is an input constraint, not optional decoration. Apparently plausible array extents or intensity ranges can still hide wrong axes, transforms, or label semantics. A sanity check should therefore use declared physical coordinates: compare known landmarks or physical extents before and after a transform, and round-trip representative indices or landmarks through the index-to-physical and physical-to-index mappings, checking that they return to the expected locations within a declared tolerance. For a mask or label image, also verify the label set and the meaning of each label, and confirm that image and labels still refer to the same anatomy. These checks test representation and bookkeeping; they do not prove biological fidelity or downstream task validity.

Modality and acquisition define the variation a model sees

Intensity values have modality- and protocol-specific meanings and limits. A one-channel CT representation is not an RGB image with a missing colour channel, and valid preprocessing depends on the modality and protocol that produced the representation. For CT, a relation to attenuation on a Hounsfield scale remains conditional on the stored values and transformations being handled correctly. The same numerical operation can therefore represent a plausible acquisition variation in one task and alter or erase the intended signal in another. The question is not whether a library exposes a transform, but what physical or clinical variation that transform is meant to model.

Acquisition and reconstruction choices create further axes of variation. Dose, contrast timing, field of view, voxel spacing, reconstruction kernel, and site can all change the distribution presented to a model, but they are not one undifferentiated nuisance variable. Some choices concern how the measurement was made; others concern how acquired information was reconstructed or represented. A model trained on one combination can encounter a different input distribution when any of those combinations changes. Internal performance on a test set that shares those conditions is evidence about that represented setting, not by itself a deployment claim. The validation chapter’s discussion of testing claims is the appropriate framework for deciding what a new setting tests.

Labels are also scarce in a way that changes the engineering problem. Many medical-imaging studies have far fewer independent patients than the number of array elements, slices, or derived patches suggests. Augmentation can help represent selected variation, but it cannot manufacture independent clinical evidence. It must also preserve the target of the task. A spatial transform needs to be applied consistently to the image and its labels, with the coordinate metadata handled according to the implementation. An intensity transform needs a stated interpretation: it may model a declared acquisition variation, but it may also change the physical quantity or erase the distinction the label depends on.

This makes universal augmentation rules unsafe. Colour jitter has no direct colour meaning for a single-channel CT representation. Arbitrary intensity scaling is not automatically a physically plausible change to a calibrated value. A 90-degree array rotation is not automatically safe or unsafe: it may be appropriate for a task whose spatial symmetries and label semantics justify it, while being implausible for an anatomical orientation task; in either case, the implementation must keep the image, labels, and geometry consistent. Small deformations, flips, rotations, or intensity perturbations are design choices whose validity depends on modality, anatomy, task, coordinate convention, and the population the model is intended to represent. No angle, flip, deformation, or intensity threshold is universally safe.

An augmentation audit makes those conditions explicit. For each proposed transform, record its intended acquisition or anatomical interpretation, its validity boundary, the required image/label/geometry operation, and a counterexample or negative-control outcome:

Transform	Intended interpretation	Validity boundary	Image/label/geometry contract	Counterexample or negative-control outcome
Colour jitter	No direct acquisition interpretation for a single-channel CT representation; normally an excluded transform or an explicit negative control.	Do not treat a library operation as physical variation or as a transport argument.	No spatial metadata or label transform makes colour jitter anatomically valid; record why it is outside the input contract.	A response to colour-only changes is a representation or pipeline warning, not evidence of an anatomical cue.
Intensity scaling or perturbation	A declared dose, noise, or contrast change only when that mapping is justified for the modality and protocol.	Arbitrary changes can alter calibrated values or erase task-relevant intensity meaning.	Record the stored-value mapping; keep labels and geometry unchanged only when the intended change is non-spatial, and verify the resulting intensity interpretation.	A calibration-sensitive task or an out-of-contract perturbation is a counterexample; behaviour there is not evidence of deployment robustness.
90-degree rotation	A declared acquisition orientation or a justified spatial symmetry of the task and population.	It is invalid for an orientation- or laterality-sensitive task unless the coordinate and label semantics explicitly support it.	Apply the same spatial transform to image and labels, then update and verify the coordinate metadata.	An orientation-sensitive landmark or label is a counterexample; use it as a negative control rather than treating a changed score as augmentation benefit.
Flip	A justified anatomical symmetry or acquisition-orientation variation.	Laterality, directional anatomy, and task labels may make a flip invalid.	Transform image, labels, and geometry under the same declared coordinate convention.	A left/right or directional label provides a counterexample; a changed result alone does not identify a site or biological cue.
Deformation	A bounded model of measured motion or anatomical shape variation.	The deformation must remain plausible for the anatomy and target population and must not change the task definition.	Co-transform image, masks, labels, and relevant geometry; check landmarks and label correspondence.	A deformation that moves landmarks outside the intended anatomy or changes label meaning is an invalid augmentation or negative control.

This mapping is part of the input contract, not a post-hoc explanation. A pipeline can respond to a site, reconstruction, or augmentation cue, but a targeted sensitivity or transport test is needed before observed behaviour is attributed to that specific cue.

A tensor pipeline can hide a deployment assumption

The risks compound. If a pipeline ignores spacing, its fixed receptive field can represent different physical extents across scans. If it also learns from one reconstruction or site, it may use texture or protocol cues that change at deployment. If the augmentation contract changes intensity meaning or label geometry, the training distribution no longer describes the intended task even before an external site is considered. These are possible failure pathways, not diagnoses of a particular model; targeted sensitivity or transport tests are needed before attributing observed behaviour to a specific site, reconstruction, or augmentation cue.

The practical response is to write the input contract before fitting: identify the modality and intensity interpretation, record geometry and reconstruction, state which acquisition axes are in scope, and define each spatial or intensity transformation in those terms. Then test the complete procedure under the changes the deployment claim actually covers. A strong internal score can be useful evidence for that internal setting, but it does not erase an untested shift in geometry, modality, acquisition, or site.

The next consequence is downstream rather than cosmetic. Once an image is used to measure a region, the segmentation mask becomes part of the measurement definition, and its quality must be judged by the task it enables.