Correlation, concordance, and agreement

A repeatability study can report an impressive correlation while the two measurements are systematically different. Take the deliberately synthetic vectors X = [1, 2, 3] and Y = [2, 4, 6]. Every value in Y is twice the corresponding value in X. The relationship is perfectly linear, but the two measurements are not interchangeable. The distinction between those statements is the reason to separate correlation from agreement.

Association is not agreement

Correlation measures how closely paired observations follow a line. Agreement asks whether they follow the line of identity, y = x. A systematic offset or scale change can leave a strong correlation intact while making a retest read consistently high or low. For a measurement that is meant to be repeatable, that calibration difference is not a detail to hide inside one correlation coefficient.

The arithmetic makes the distinction visible. For X, the mean is (1 + 2 + 3) / 3 = 2; for Y, it is (2 + 4 + 6) / 3 = 4. Their means already show a systematic shift. The deviations from the means are [-1, 0, +1] and [-2, 0, +2]. The vectors have the same pattern around their centres, but their spread is different.

From moments to Pearson correlation

For this worked derivation, use population moments over the three observed pairs. The variance and covariance are

var(X) = sum((xi - mean(X))^2) / N = (1 + 0 + 1) / 3 = 2/3
var(Y) = (4 + 0 + 4) / 3 = 8/3
cov(X,Y) = sum((xi - mean(X))(yi - mean(Y))) / N = 4/3

Pearson correlation cancels the units of covariance:

r = cov(X,Y) / sqrt(var(X) * var(Y))
  = (4/3) / sqrt((2/3)(8/3))
  = 1.00

The result is correct for the question Pearson’s r answers. More generally, Y = a + bX with b > 0 gives r = 1 for any offset a and positive scale b. It says that the observations have a perfect positive linear association; it does not say that the readings agree in value.

Agreement with the line of identity

Lin’s concordance correlation coefficient (CCC) adds the calibration question to the correlation question. Its reference is the 45-degree line through the origin, the line on which the two readings are equal. In a common population- moment form,

             2 * cov(X,Y)
CCC = -------------------------------------
       var(X) + var(Y) + (mean(X) - mean(Y))^2

The denominator contains the shared variation’s competitors: the spread of each vector and the squared difference of their means. The covariance supplies the shared variation in the numerator. CCC can be understood as a measure that combines precision, represented by correlation, with a penalty for location and scale disagreement. It is designed for agreement with identity, not merely for finding any line that fits the points.

For the synthetic vectors,

CCC = (2 * 4/3) / (2/3 + 8/3 + (2 - 4)^2)
    = (8/3) / (22/3)
    = 4/11 = 0.3636...

Thus r = 1.00 and CCC = 4/11. The correlation sees a perfect line; the concordance sees that the line is y = 2x, not y = x. In the identity control Y = [1, 2, 3], both coefficients are 1. In the reflection control Y = [3, 2, 1], the correlation is -1 and the CCC is -1: the measurements move in opposite directions around the same mean.

Synthetic vectors for definitions only, not measurements of a biomarker. These CCC values are not quality thresholds or clinical benchmarks. The estimator convention is part of the statistical contract, and an agreement coefficient does not by itself describe an absolute measurement error in the feature’s units.

Estimator conventions and undefined cases

Dividing by N and dividing by N - 1 answer different variance questions. For the same Y = 2X example, the N-moment calculation gives CCC = 4/11, while using sample moments gives var(X) = 1, var(Y) = 4, and cov(X,Y) = 2, hence CCC = 4/9. The arithmetic is not an argument that one convention is always right. It is an argument that the convention must be stated before results are compared. Pearson’s common scaling factors cancel between numerator and denominator; they do not cancel in CCC in the same way.

The coefficient also has genuine edge cases. With fewer than two finite paired observations, it is undefined. If both vectors are constant and equal, the denominator is zero: there is no variation from which to assess concordance, so the result must remain undefined rather than being silently changed to 1. If both vectors are constant but differ, the denominator is nonzero and the CCC is 0. These cases should be reported as mathematical or data-quality outcomes, not replaced by a convenient score.

Finally, CCC is relative to the variation in the paired cohort. The same absolute scan-to-scan difference can produce different CCC values in a narrow and a wide range of subjects. CCC is therefore not an absolute error model and it is not a universal pass/fail threshold. Its interval, paired differences, and the scatter around the identity line all belong in a repeatability report.

The next section asks how to put uncertainty around such a coefficient when the cohort is finite: resampling must preserve the subject pair.

Sources

• Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45(1):255-268. doi:10.2307/2532051, PMID 2720055.
• Lane DM, Online Statistics Education, “Values of the Pearson Correlation” — open textbook.
• Ye P, et al. BJR Open. 2022;4(1):20210083. doi:10.1259/bjro.20210083, PMID 36452056.