Read once for the story, then reconstruct the study

Tonight · ~20 min · read · energy: low · setup: one paper to practise on

The first pass of a paper is for the story; the second is for the structure. This lesson is the reconstruction step that turns a narrative paper into a finite list of study-design facts you can audit. Once you can reconstruct a study in one paragraph, every later lesson has something to bite into.

The anchor: a paper is a story built on a study

Papers are written to persuade, which means the design is spread across abstract, methods and results in a narrative order that hides the structure. To appraise it, you must rebuild the skeleton: what question, in whom, with what inputs, predicting what outcome, split how, claiming what. If you cannot write that skeleton in one paragraph, you have not understood the study — and neither, often, has the abstract.

The reconstruction paragraph

Read the paper once for flow, then reconstruct these six in one paragraph:

Question — the precise clinical question (prediction vs diagnosis vs prognosis vs segmentation). Methods must match the question.
Population — who is in/out and why; exclusions that remove hard cases inflate performance (spectrum bias).
Inputs — what the model actually sees (which series, which reconstruction, which contrast phase — Ch. 1).
Outcome — the label being predicted, and where it came from (lesson 3).
Split — how train/val/test were formed, and the unit of splitting (Ch. 4, lesson 4).
Claim — what the authors conclude, and how wide that claim is relative to the evidence (lesson 6).

Why this is the gating step

Every later lesson interrogates one of these six. “Did the test set leave the development world?” presupposes you know the split. “Do the metrics support the claim?” presupposes you know the claim. So reconstruction is not busywork — it is the prerequisite that makes the rest of the checklist applicable. A paper whose design you cannot reconstruct is a paper you cannot appraise; that itself is a finding (methods under-specified).

A worked mini-reconstruction

Take Tran et al. (PMID 35726103) — we dissect it fully in lesson 8, but the one-paragraph reconstruction is: detect ACL tear on knee MRI (question); ~19,800 knee MRI studies across 12 centres, multi-vendor, multi-field (population); the MRI examination (input); ACL tear, labelled from radiology reports via an NLP model (outcome — note the reference standard, lesson 3); 70/20/10 patient-disjoint train/val/test plus two external datasets (split — note the frozen-vs-retrained issue, lesson 4); multi-continental external validation (claim — which lesson 6 and the dissection will qualify). Six lines, and already the audit hooks are visible.

Stop and think — then reveal

You read a paper’s abstract and conclusion and feel convinced. Why is stopping there dangerous, and what is the first thing you should write down before reading further?

The abstract and conclusion are the most persuasive, least auditable parts of a paper — they state claims, not the design that supports them. Stopping there means accepting the authors’ framing without seeing the population, inputs, reference standard, split or metrics. The first thing to write down is the reconstruction paragraph (the six items above). The moment you try to write it, gaps surface — an under-specified split, an unclear reference standard, inputs pooled across reconstructions — and those gaps are the appraisal. If the paragraph will not write cleanly from the paper, the methods are under-specified, which is itself a red flag.

What to retain

Read once for the story, then reconstruct the study’s skeleton in one paragraph.
Six items: question, population, inputs, outcome, split, claim. Every later lesson interrogates one of them.
If you cannot reconstruct the design, you cannot appraise the paper — and under-specified methods are themselves a finding.
The reconstruction surfaces the audit hooks (reference standard, frozen-vs- retrained, overclaim) before you read the details.

Next: the first item in depth — what exactly is the unit of analysis?