Correlation and Linear Regression

Correlation tells you how strongly two variables move together, while linear regression gives you an equation to predict one from the other. Both appear constantly in FSc, A Level and university statistics, and both reduce to a handful of sums you can organise neatly. This guide explains the formulas and works through two complete examples.

What Correlation Measures

The Pearson correlation coefficient \( r \) measures the strength and direction of a linear relationship between two variables \( x \) and \( y \). It always lies between \(-1\) and \(1\):

• \( r = 1 \): perfect positive linear relationship.
• \( r = -1 \): perfect negative linear relationship.
• \( r = 0 \): no linear relationship (though a curved one might exist).
• Values like \( 0.8 \) or \( -0.8 \) are usually called strong; values near \( 0.3 \) are weak.

The Key Sums

Everything depends on three quantities, built from deviations about the means \( \bar{x} \) and \( \bar{y} \):

$$ S_{xx} = \sum (x - \bar{x})^2, \quad S_{yy} = \sum (y - \bar{y})^2, \quad S_{xy} = \sum (x - \bar{x})(y - \bar{y}) $$

Equivalent shortcut forms are \( S_{xx} = \sum x^2 - \frac{(\sum x)^2}{n} \) and \( S_{xy} = \sum xy - \frac{\sum x \sum y}{n} \).

The correlation coefficient is

$$ r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} $$

The Least Squares Regression Line

The regression line of \( y \) on \( x \) is the straight line \( \hat{y} = a + bx \) that minimises the sum of squared vertical distances from the points to the line. Its coefficients are

$$ b = \frac{S_{xy}}{S_{xx}}, \qquad a = \bar{y} - b\bar{x} $$

Two useful facts:

• The line always passes through the point \( (\bar{x}, \bar{y}) \).
• The slope \( b \) and the correlation \( r \) always have the same sign.

The Coefficient of Determination

The value \( r^2 \) tells you the proportion of the variation in \( y \) that is explained by the linear relationship with \( x \). For example, \( r^2 = 0.6 \) means 60% of the variation is explained.

Worked Example 1: Positive Correlation

The data below show hours of revision \( x \) and test score \( y \) (out of 10) for five students.

Step 1: Means.

$$ \bar{x} = \frac{1+2+3+4+5}{5} = 3, \qquad \bar{y} = \frac{2+4+5+4+5}{5} = \frac{20}{5} = 4 $$

Step 2: The sums.

$$ S_{xx} = 4 + 1 + 0 + 1 + 4 = 10 $$

$$ S_{yy} = 4 + 0 + 1 + 0 + 1 = 6 $$

$$ S_{xy} = (-2)(-2) + (-1)(0) + (0)(1) + (1)(0) + (2)(1) = 4 + 0 + 0 + 0 + 2 = 6 $$

Step 3: Correlation.

$$ r = \frac{6}{\sqrt{10 \times 6}} = \frac{6}{\sqrt{60}} \approx \frac{6}{7.746} \approx 0.775 $$

This is a fairly strong positive correlation.

Step 4: Regression line.

$$ b = \frac{6}{10} = 0.6, \qquad a = 4 - 0.6 \times 3 = 4 - 1.8 = 2.2 $$

$$ \hat{y} = 2.2 + 0.6x $$

Step 5: Interpret and predict. Each extra hour of revision is associated with a 0.6-mark increase. For \( x = 3.5 \) hours, \( \hat{y} = 2.2 + 0.6(3.5) = 2.2 + 2.1 = 4.3 \). Also \( r^2 = 36/60 = 0.6 \), so 60% of the variation in scores is explained by revision time.

Worked Example 2: Negative Correlation

Four students record daily hours of phone use \( x \) and a quiz score \( y \): \( (2, 10), (4, 9), (6, 6), (8, 3) \).

Step 1: Means.

$$ \bar{x} = \frac{2+4+6+8}{4} = 5, \qquad \bar{y} = \frac{10+9+6+3}{4} = \frac{28}{4} = 7 $$

Step 2: Deviations. \( x - \bar{x} \): \( -3, -1, 1, 3 \). \( y - \bar{y} \): \( 3, 2, -1, -4 \).

Step 3: The sums.

$$ S_{xx} = 9 + 1 + 1 + 9 = 20 $$

$$ S_{yy} = 9 + 4 + 1 + 16 = 30 $$

$$ S_{xy} = (-3)(3) + (-1)(2) + (1)(-1) + (3)(-4) = -9 - 2 - 1 - 12 = -24 $$

Step 4: Correlation.

$$ r = \frac{-24}{\sqrt{20 \times 30}} = \frac{-24}{\sqrt{600}} \approx \frac{-24}{24.495} \approx -0.980 $$

This is a very strong negative correlation.

Step 5: Regression line.

$$ b = \frac{-24}{20} = -1.2, \qquad a = 7 - (-1.2)(5) = 7 + 6 = 13 $$

$$ \hat{y} = 13 - 1.2x $$

Each extra hour of phone use is associated with a 1.2-point drop in quiz score. For \( x = 5 \), \( \hat{y} = 13 - 6 = 7 \), which matches \( \bar{y} \), as it must since the line passes through \( (\bar{x}, \bar{y}) \).

Common Mistakes

• Confusing correlation with causation. A strong \( r \) does not prove that \( x \) causes \( y \). A third factor may drive both.
• Extrapolating. Predicting far outside the range of your data (for example, 20 hours of revision above) is unreliable.
• Using \( r \) for curved data. A perfect parabola can have \( r = 0 \). Always sketch a scatter diagram first.
• Mixing up the regression lines. The line of \( y \) on \( x \) is for predicting \( y \). Predicting \( x \) from \( y \) requires a different line, \( b' = S_{xy}/S_{yy} \).
• Rounding too early. Keep full sums until the final step.

Key Takeaways

• \( r = S_{xy}/\sqrt{S_{xx}S_{yy}} \) measures the strength and direction of a linear relationship, from \(-1\) to \(1\).
• The least squares line is \( \hat{y} = a + bx \) with \( b = S_{xy}/S_{xx} \) and \( a = \bar{y} - b\bar{x} \).
• The regression line always passes through \( (\bar{x}, \bar{y}) \).
• \( r^2 \) gives the proportion of variation in \( y \) explained by \( x \).
• Correlation is not causation, and predictions are only trustworthy within the data range.

Have a regression dataset or statistics assignment you need help with? Message Amathify on WhatsApp or use the "Ask a Question" form, and our experts and PhD scholars will send you a complete step-by-step solution.