Case study: missing data and imputation

A missing value is part of the problem, not just a hole

A clinical table can contain an observed subject with an unrecorded field, or it can omit a subject entirely. Those are different sources of uncertainty. A missing histology value concerns the field and the process by which it was not recorded. An absent subject concerns cohort construction and selection. Filling the first does not repair the second.

The missingness mechanism is usually described with three labels. Under MCAR, missingness is unrelated to observed and unobserved values under the stated model. Under MAR, missingness may depend on variables that were observed. Under MNAR, missingness depends in part on an unobserved value or another unobserved process. These are assumptions about how the data arose, not labels that the observed table can prove by itself. An observed predictor of missingness is evidence against MCAR for that predictor and is compatible with MAR; finding no such predictor does not prove MCAR or rule out MNAR.

The mechanism and the imputation method are separate decisions. Under suitable conditions relating the missingness mechanism to the analysis model, complete- case analysis may target a stated estimand without bias, but MCAR alone is not a blanket guarantee for every analysis. Complete-case analysis discards observations and loses precision. Mean or mode imputation creates one value for many different subjects and can distort variance, covariance, and uncertainty even when the missingness mechanism is benign. Under MAR, observed variables can support a conditional imputation model, but the assumption remains a modelling argument. Under MNAR, an imputation model alone does not identify the missing values; sensitivity analysis or a different data-collection design is needed.

Artificial masking measures a different quantity

Natural missing cells have no observed truth against which to score a filled value. A bounded method comparison can therefore start with cells that are observed, hide a declared subset, and compare the imputed values with the original values. This evaluates reconstruction under the artificial masking mechanism. It does not prove that the same method is correct for the natural missingness process.

The design has four important controls. First, profile natural missingness separately rather than treating it as ground truth. Second, mask only observed predictor cells according to a declared scheme. Third, use the same frozen masks for every method so that methods face the same targets. Fourth, score only on the masked cells whose original values are known. Outcomes, follow-up fields, and identifiers should not become imputer inputs merely because they improve the reconstruction score; whether an auxiliary variable is available at deployment must be declared before fitting.

Artificial masking is therefore an evaluation instrument. Its result is a method-comparison estimand: how do the candidate imputers behave under these masks, fields, split rules, and scoring metrics? It is not an estimate of the unobserved clinical truth for the naturally missing rows.

A methods ladder keeps the baseline visible

A simple median or mode imputer is a useful baseline when fitted on training data only. It is not a claim that the filled value is the subject’s true value, and its distortion of variation is part of why more complete methods may be considered. A sophisticated imputer that cannot improve on this baseline under the declared evaluation has not earned its complexity.

A predictive imputer models one incomplete field from the other permitted fields. A project may use one model per target field, with separate handling for continuous and categorical targets. The exact algorithm, configuration, and availability of predictors are part of that project contract; they are not universal properties of imputation.

Multiple imputation has a different objective from producing one convenient table. In a chained-equation framework, a conditional model is specified for each incomplete variable, the variables are updated in sweeps, and multiple completed datasets represent uncertainty about the missing values. The analysis then combines estimates across those datasets with within- and between- imputation variation. A single completed table cannot be described as full multiple imputation merely because its conditional models were applied in a round-robin order.

That distinction matters in the dated case below. The project used a documented single-table chained-equation proxy based on an iterative imputer and a Bayesian linear model, not multiple completed datasets with pooled inference. Calling the proxy canonical MICE would overstate what its uncertainty treatment can establish.

Imputation does not recover ground truth

An imputed value is model output conditional on the observed fields and the missingness assumptions. It is not a delayed measurement. A conditional mean can be a reasonable prediction for one task and a poor representation of a subject whose value was missing for an unobserved reason. The clinical measurement layer still matters: before imputing, state what each field represents, when it was available, how it was measured, and whether the deployment setting would have it at the same time.

Imputation can also change the downstream estimand. A model trained on a table where missing values were dropped answers a question about complete cases. A model trained after single imputation answers a question about a transformed table. A multiple-imputation analysis can propagate imputation uncertainty into the target estimate, but only if the imputation and pooling procedure match the target analysis.

Splits and fitted states make the comparison meaningful

The imputer is part of the predictive pipeline. The split and grouping rule must match the target. When repeated records belong to the same subject or another cluster and the claim concerns new subjects or clusters, make the development, validation, and test split at that cluster level. A within-subject comparison may use a different grouping design, but it must be stated rather than hidden under “subject-level” language. Each imputer is fitted on the permitted training data and then applied to validation or test rows. A median computed from the complete cohort, or a predictive imputer trained on the test rows, allows the evaluation data to shape the values being scored.

The same information rule applies to mask generation, hyperparameter choice, feature selection, and any downstream survival or classification model. The outcome and identifiers are excluded from imputer inputs unless the estimand and deployment time explicitly make them available. Membership details can remain private while the aggregate split and mask provenance are recorded well enough to audit the design.

What the dated M1-07 case measured

The M1-07 Plan-B case is retained as a dated, bounded method comparison. Its aggregate provenance is recorded in the public project record at the pinned commit; that link is provenance for this case, not evidence for a general clinical claim. Its NSCLC-Radiogenomics table contained 144 subjects, 51 observed death events, and 93 censored outcomes, with two continuous and five categorical predictor fields. The frozen subject-level split assigned 101 subjects to training, 22 to validation, and 21 to testing. The evaluation masked observed predictor cells at 10%, 20%, and 30% under an MCAR construction, plus one observed-variable- dependent scenario using the declared gender-dependent probabilities. The same masks were used across methods and seeds.

The comparison included median/mode, a single-table chained-equation proxy, and target-wise predictive imputation. The project record reports 60 of 60 planned runs completed without failure, with reconstruction metrics on held-out masked cells and dependency-preservation summaries calculated under the frozen design. Those numbers establish what that project executed and compared. They do not establish clinical validity, prove the natural missingness mechanism, or show that one method is best for every clinical table.

What the case can and cannot show

The case can show how the candidate methods behaved when the same known cells were artificially hidden, under a subject-level split and a declared input schema. It can reveal whether a method’s apparent advantage depends on a mask scenario, a field type, a seed, or a dependency metric. It can also make the single-table proxy versus canonical multiple-imputation distinction visible.

It cannot turn artificially masked cells into a verified sample of the natural missing values. It cannot test whether MAR or MNAR is true from the observed table alone. It cannot convert an imputed clinical field into a direct measurement, and it cannot make a downstream model clinically valid without a separate outcome and transport design.

The important negative result is methodological rather than rhetorical: a filled table is not evidence that the missingness problem has disappeared. A reader should be able to reconstruct the missingness assumptions, mask targets, split boundary, imputer state, and downstream use without opening a patient row or relying on the project’s internal chronology.

This case therefore follows directly from the protocol-first section. The next question changes the task: instead of filling fields for existing subjects, what does it mean to generate complete synthetic rows, and how can utility and risk be kept separate?

Sources

• van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software. 2011;45(3):1-67. doi:10.18637/jss.v045.i03.
• Austin PC, White IR, Lee DS, van Buuren S. Missing Data in Clinical Research: A Tutorial on Multiple Imputation. Canadian Journal of Cardiology. 2021;37:1322-1331. doi:10.1016/j.cjca.2020.11.010; PMC8499698.
• Prokhorenkova L, et al. CatBoost: unbiased boosting with categorical features. NeurIPS 2018. arXiv:1706.09516.