Correlation and regression

Correlation: do they move together?

The correlation coefficient, r, measures how closely two variables follow a straight line. It runs from −1 (a perfect line sloping down) through 0 (no straight-line relationship) to +1 (a perfect line sloping up). It has no units, so it does not change if you measure in millimeters or inches.

Rough descriptions: above about 0.8 is strong, 0.5 to 0.8 moderate, 0.3 to 0.5 weak. These are conventions, not rules. The tool also tests whether r is statistically significant, because with few points a large r can appear by chance.

Regression: the line itself

Simple linear regression fits the straight line y = b₀ + b₁x that minimizes the squared vertical distances from the points. The slope b₁ says how much y changes, on average, for each unit of x. r² is the share of the variation in y that the line accounts for: r = 0.9 gives r² = 0.81, so 81% is explained and 19% is not.

Three cautions

• Correlation is not causation. Ice-cream sales and drownings correlate because both rise in summer. To show that x drives y, change x on purpose and watch y.
• Do not extrapolate. The line is only supported over the range of x in the data. The tool warns when you predict outside it.
• Look at the plot. r measures straight-line relationships only. A clear curve can have r near zero, and one outlier can create or hide a correlation.

How sure is r? The Fisher z interval

The sampling distribution of r is skewed, especially near ±1, so r ± a margin does not work. Fisher's transformation z = ½ ln((1 + r)/(1 − r)) is close to normal with standard error 1/√(n − 3). The tool builds the interval z ± z_α/2/√(n − 3) and turns both ends back into r. If the interval for ρ includes 0, the data are consistent with no linear relationship. With 14 pairs, even r = 0.98 leaves an interval about 0.05 wide.

Confidence interval or prediction interval?

At a given x, the confidence interval is for the average y of all items at that x: ŷ ± t·s·√(1/n + (x − x̄)²/Sxx). The prediction interval is for one new item: ŷ ± t·s·√(1 + 1/n + (x − x̄)²/Sxx). The extra 1 under the root is the item's own scatter around the line, so the prediction interval is always wider and does not shrink to zero as n grows. Both are narrowest at x̄ and flare toward the ends, as the bands on the scatter diagram show. The tool also gives the interval and the t test for the slope, with t on n − 2 degrees of freedom.

Residual analysis

The intervals and tests assume a straight-line relationship with independent, normal errors of constant variance. The residuals (y − ŷ) check that. On residuals versus fitted values, a curve means the straight line is the wrong model and a funnel means the variance is not constant. On the normal probability plot, points near the line support the normal assumption. The table gives each point's standardized residual (beyond ±2 is worth a look, beyond ±3 is a likely outlier) and its leverage (how far its x is from the rest, so how hard it pulls the line).

On the exam

Correlation (IV.F.1) and regression (IV.F.2) are in the 2022 CSSYB Body of Knowledge: read r from a description, interpret r², and spot a causation claim the data cannot support. The CQE adds hypothesis tests and prediction with regression (VI.E.1) and the confidence interval for the correlation coefficient (VI.E.2). The CSSBB asks for the correlation coefficient and its confidence interval (VI.A.1), and for estimation, the uncertainty in the estimate and a residuals analysis to validate the model (VI.A.2). CQPA III.F.1 covers how regression and correlation models are used for estimation and prediction.

Other tools and templates