TRACE-CT learning map

TRACE-CT (local ../trace-ct when checked out; Forgejo BioMedical-IT/trace-ct — the develop branch holds the current state) is the driving laboratory. This map connects its real work to the learning chapters so the path is never abstract. All paths below are read-only references — TRACE-CT code is not copied into this repo (D-001).

Status vocabulary: MEASURED = executed with committed machine-readable evidence. IMPLEMENTED / FROZEN DESIGN = code/config/statistics exist and are frozen, but no full-cohort result exists yet. Never present a frozen design as a measured result.

TRACE-CT work → learning chapters

Foundations (MEASURED, unchanged)

TRACE-CT work	Concept learned	Chapter	Why it matters
RIDER inventory & preflight — scripts/rider_preflight.py (scan, classify CT/SEG, group acquisitions, select reconstructions, pair repeats, resolve SEG refs, evaluate geometry, assess usability)	DICOM object classification; Study/Series grouping; reconstruction selection; test–retest pairing	1, 2	Real patient-level data handling; the data boundary in practice
DICOM metadata inspection — rider_preflight.py (Series/Study/FoR UIDs, kernel, kVp, SliceThickness, ImageType, PixelSpacing, IOP/IPP)	The DICOM hierarchy & identifiers; kernel/thickness as variables	1	The contract (metadata) you must read before the payload (pixels)
CT/SEG linkage — build_seg_meta() / resolve_seg_references(); seg_decode_resample.py decode_seg_file() / build_source_mask_on_ct_grid()	Source-image references; per-frame functional groups; segment identity	2	How a SEG names its source CT and why matching is by SOP UID
Geometry validation — compute_slice_normal(), build_affine(), project_position(), orientations_equivalent(), spacings_equivalent(), positions_equivalent(); load_ct_volume_explicit()	Direction cosines; slice normal; geometric ordering; EXACT_GRID vs RESAMPLED	2	The whole Part A/B of Ch. 2, implemented and tested
HU conversion — load_ct_volume_explicit() (pixel*slope+intercept); clip_intensities() (clip [-1000,+400] after resampling)	RescaleSlope/Intercept; working in physical HU; clip-after-resample order	1, 2, 3	Never use stored pixels as HU; operation order is part of the measurement definition
Mask transfer & resampling — resample_mask_inverse_nn(), classify_transfer_pair(), verify_determinism(); resample_image_isotropic() (B-spline → 1 mm), resample_mask_nn(), affine_to_sitk_geometry()	Linear vs nearest-neighbour; inverse NN; determinism; SimpleITK bridge; B-spline overshoot + clip	2	Masks resample differently from images; same shape ≠ same space
Visual QC in 3D Slicer — tools/slicer/open_qc_bundle.py; scripts/visual_qc/*; schemas/visual-qc-bundle-v1.schema.json; aggregate_qc_reviews.py (privacy-safe aggregate review evidence)	Visual QC as real, auditable geometry/overlay verification	2, 3	See the geometry concepts in a viewer; record a defensible verdict
Determinism / independent-process comparison — check_determinism=True (1e-6); compare_radiomics_runs.py	Reproducibility of the numbers before any statistics	3, 4	Agreement within the declared numerical tolerance is not bitwise identity; exact identity requires a separate check

Preflight (rider_preflight.py check_geometry()) checks selected-series spatial metadata completeness and SEG linkage and assigns a subject-level geometry state using EXACT_GRID, RESAMPLING_REQUIRED_UNVALIDATED, INCOMPATIBLE, or UNRESOLVED. The qualification matrix further distinguishes validated resampling and missing cases (RESAMPLING_REQUIRED_VALIDATED, MISSING). Separately, the geometry helpers in seg_decode_resample.py compare orientation, spacing, and positions within their configured tolerances, and classify_transfer_pair() summarizes a source-to-target transfer pair as exact_grid or resampled. A preflight state alone is not proof of physical-grid compatibility. The book’s EXACT_GRID/RESAMPLING_REQUIRED/INCOMPATIBLE labels are chapter teaching labels, not the project’s internal state enum.

M1-07 — clinical missingness & imputation (MEASURED)

TRACE-CT artifact (read-only)	Concept learned	Chapter
configs/m1-07-plan-b.json + results/metrics/m1-07-plan-b-split-mask-provenance.json	One frozen subject-level split (101/22/21, event×gender stratified, SHA-256 membership hash); frozen-seed masks	4 (lesson 9)
scripts/m1_07_experiment.py; results/metrics/m1-07-plan-b-imputation-metrics.json	Median/mode baseline; MICE-family proxy (IterativeImputer+BayesianRidge — not full multiple imputation); target-wise CatBoost; artificial masking (MCAR 10/20/30% + gender-dependent); train-only fitting; outcomes/identifiers structurally excluded	4 (lesson 9)
results/metrics/nsclc-radiogenomics-m1-07-clinical-profile.json	Missingness as a measured variable; the clinical baseline imaging must beat	4
Measured status	60/60 runs, zero failures; held-out reconstruction + train-only dependency metrics per run — a reproducible method comparison, not a clinical-validity claim	4

M1-08 — bounded generative experiment (MEASURED)

TRACE-CT artifact (read-only)	Concept learned	Chapter
configs/m1-08-plan-b.json	Frozen generated schema (9 fields), complete-case N=94 generator reference, 51-row Female deficit, seeds 1729/2718/3141, one bounded adjustment per model	4 (lesson 10)
scripts/m1_08_generative_experiment.py	CTGAN conditional generation vs unconditional TabDDPM + generation-order selection (never “conditional TabDDPM”); no quality-filtering; no real-row downsampling	4 (lesson 10)
results/metrics/m1-08-plan-b-generative-metrics.json + results/figures/m1-08/	Per-field fidelity (KS/prevalence), dependency preservation (Pearson/Spearman · Cramér’s V · η), Gower memorisation-risk indicators, subgroup reportability gates	4 (lesson 10)
Downstream Cox evaluation	Clinical-only reference C-index 0.789 vs TabDDPM-augmented 0.743–0.817 (CI-overlapping — no demonstrated gain); calibration/IBS only with ≥ 10 events; retained negatives (CTGAN conditional mapping unproven; no KPI1-C passes; elevated TabDDPM memorisation signals 2/3 seeds)	4 (lessons 7, 10)
docs/model-card-m1-08.md	Honest reporting of a bounded generative experiment, negatives included	4, 7

M1-09 — RIDER radiomics robustness (M1-09A: IMPLEMENTED / FROZEN DESIGN; M1-09B: MEASURED qualification; M1-09C: MEASURED primary test–retest repeatability; M1-09D/E: pending/follow-up)

The M1-09A freeze (D-023, merged 19 Aug 2026; issues #49/#48) is the richest single learning artifact in TRACE-CT: an entire robustness study specified before any full-cohort result existed. M1-09B is now measured full-cohort qualification evidence (TRACE-CT CL-025, rider-m1-09b-qualification.json), and the frozen primary M1-09C test–retest analysis is measured under the frozen contract. These results are evidence for their cohort and protocol, not a universal claim about feature families or clinical utility; M1-09D/E remain pending follow-up work.

Frozen artifact (read-only)	Concept learned	Chapter
Protocol: docs/experiment-protocol.md § M1-09 + DECISIONS.md D-023	Pre-specification / anti-tuning: freeze cohort, preprocessing, statistics, thresholds before results; anti-drift applied to one’s own protocol	3 (lesson 13), 7
Six-setting design (classify_source_role() tags lung_1_25mm vs standard_5_0mm): 1.25L/1.25S/2.5L/2.5S/5L/5S; primary = 1.25L test–retest; secondary = all 15 setting pairs	Reconstruction as a matrix, not an anecdote; repeatability vs reconstruction agreement as distinct analyses	1 (lesson 3), 3 (lesson 6)
Preprocessing contract — scripts/rider_radiomics_extraction.py: CT → SEG decode → external 1 mm B-spline/NN resample → clip [-1000,+400] → PyRadiomics; no-double-resampling rule	Operation order as part of the measurement definition; clipping ≠ resegmentation	2 (lesson 7), 3 (lesson 10)
configs/rider-m1-09-feature-manifest.json (107 Original 3D features, SHA-256 d236c898…c5c1) + configs/pyradiomics_params.yaml	Feature families; Original-only baseline design; hashed manifest; canonical names	3 (lessons 2, 10; appendix 91)
scripts/rider_feature_manifest.py --verify	Config text vs effective configuration through the pinned extractor; version pinning (3.1.1.dev111+g8ed579383, commit 8ed579383…7522ab)	3 (lesson 10)
scripts/repeatability_stats.py + tests/unit/test_repeatability_stats.py	Lin CCC (population-moment estimator, line of identity, degenerate cases); paired-subject bootstrap (10,000, seed 1729, percentile CI, ≥ 0.95 valid, RUN/NOT_RUN); validation against analytic vectors	3 (lessons 11, 12)
Reportability rules (REPORTABLE requires ≥ 25 finite valid test–retest pairs and a mathematically defined CCC; NOT_REPORTABLE reasons: insufficient_valid_pairs_below_25, ccc_undefined_too_few_finite_pairs, ccc_undefined_degenerate_denominator, insufficient_valid_bootstrap_fraction_below_0.95, bootstrap_not_run_point_ccc_not_reportable) + CCC ≥ 0.90 descriptive benchmark	Thresholds as conventions; descriptive ≠ pass/fail ≠ feature selection; eligible-N semantics (failures reduce finite pairs, never eligible counts)	3 (lessons 6, 11)
Guardrails: no ComBat/harmonisation/wavelets/denoising/selection/modelling in M1-09	The measurement layer; why measure baseline before intervening; one experiment, one question	3 (lessons 9, 13)

Execution sequence (current status): M1-09B is MEASURED full-cohort qualification evidence (CL-025 / rider-m1-09b-qualification.json) → M1-09C is MEASURED primary 1.25L test–retest repeatability → M1-09D/E remain pending/follow-up work for reconstruction-setting comparisons and the evidence package. M1-09C does not answer reconstruction robustness, establish a universal feature-family ranking, or establish clinical validity.

Future learning opportunities (not yet active)

Opportunity	Concept	Chapter	Trigger / constraint
M1-09D/E measured results	Reading frozen-design-then-measured evidence as it lands	3, 4	M1-09D–E execution
CT harmonisation (ComBat)	Scanner/site batch-effect correction & its assumptions — after baseline robustness, as a separately frozen experiment	3, 4	M1-09 measured; respects ComBat’s assumptions
Missing-acquisition image synthesis	Conditional image synthesis; beyond-pixel-realism evaluation	6, note	TRACE-CT activates synthesis (D-011); current pre-Spark experiment stays bounded

How to use this map

• When a chapter says “TRACE-CT connection”, the concrete path is here.
• Open the TRACE-CT file next to the chapter; read it to see the concept implemented, never to copy it here.
• Distinguish MEASURED from IMPLEMENTED / FROZEN DESIGN in everything you write — the map’s status column is part of the lesson.
• Future opportunities are not commitments — they activate only on a project trigger (anti-drift), and never expand the bounded current experiment.