Least-Squares Regression

AP Statistics· 12 min read

1. Core Definition of the Least-Squares Criterion★★☆☆☆⏱ 3 min

Unlike arbitrary manually fitted lines, the least-squares regression line is mathematically optimized to minimize the sum of squared vertical distances between each observed data point and the line. Squaring residuals penalizes larger deviations more heavily, preventing extreme outliers from skewing the model disproportionately.

📘 Definition

Least-Squares Regression Line

\hat{y} = b_0 + b_1 x

The unique linear model that minimizes the value of $\sum (y_{i} - \overset{y}{^}_{i})^{2}$ , the total sum of squared residuals across all data points.

📐 Worked Example

Given summary statistics for study time (x, minutes) vs exam score (y, points): $\overset{x}{ˉ} = 45$ , $s_{x} = 12$ , $\overset{y}{ˉ} = 78$ , $s_{y} = 15$ , $r = 0.72$ , calculate the full LSRL equation.

1
First calculate the slope $b_{1}$ using the correlation formula:
2
$b_1 = r \cdot \frac{s_y}{s_x} = 0.72 \cdot \frac{15}{12} = 0.9$
3
Next calculate the y-intercept $b_{0}$ using the point of means:
4
$b_0 = \bar{y} - b_1 \bar{x} = 78 - (0.9 \cdot 45) = 78 - 40.5 = 37.5$
5
Final LSRL equation: $\overset{y}{^} = 37.5 + 0.9 x$

✓ Quick check

Test your understanding of the least-squares criterion:

What quantity does the LSRL minimize?
- Sum of residuals
- Sum of squared residuals
- Sum of absolute deviations
- Correlation coefficient
Reveal answer
Sum of squared residuals —
The least-squares criterion explicitly minimizes the total of squared vertical deviations between observed and predicted values.

Exam tip:

AP graders will deduct points if you define LSRL as minimizing "distance from points to the line" — you must specify vertical, squared deviations.

2. Interpreting Coefficients and Residuals★★★☆☆⏱ 3 min

Every coefficient interpretation on the AP exam must include full context of the variables, units, and explicit reference to predicted values rather than causal effects. Residuals tell you how far individual observed points lie above or below the model's prediction.

📐 Worked Example

For the LSRL $\overset{y}{^} = 37.5 + 0.9 x$ predicting exam score from study time, interpret the slope, then calculate the residual for a student who studied 50 minutes and scored 82 points.

1
Slope interpretation: For every additional 1 minute of study time, the predicted exam score increases by 0.9 points, on average.
2
Calculate predicted score for x=50:
3
$\hat{y} = 37.5 + 0.9(50) = 82.5$
4
Calculate residual as observed y minus predicted ŷ:
5
$e = 82 - 82.5 = -0.5$
6
Interpret residual: This student scored 0.5 points lower than the model predicted for their study time.

3. Key Mathematical Properties of LSRL★★★☆☆⏱ 3 min

The LSRL always passes through the point of means $(\overset{x}{ˉ}, \overset{y}{ˉ})$
The sum of all residuals for the full dataset equals exactly 0
The correlation coefficient r describes how tightly points cluster around the LSRL
The slope of the LSRL has the same sign as the correlation coefficient r

Line Type	Sum of Squared Residuals	Passes Through $(\overset{x}{ˉ}, \overset{y}{ˉ})$
Arbitrary manually fitted line	Larger than LSRL value	Not guaranteed
LSRL	Minimum possible value	Guaranteed

📐 Worked Example

Verify that the sum of residuals equals 0 for 3 data points: (30, 65), (45,78), (60,92) using the LSRL $\overset{y}{^} = 37.5 + 0.9 x$ .

1
Calculate predicted values for each x: $\overset{y}{^}_{1} = 64.5$ , $\overset{y}{^}_{2} = 78$ , $\overset{y}{^}_{3} = 91.5$
2
Calculate residuals: $e_{1} = 65 - 64.5 = 0.5$ , $e_{2} = 78 - 78 = 0$ , $e_{3} = 92 - 91.5 = 0.5$
3
Sum residuals: $0.5 + 0 + (- 0.5) = 0$

4. Limitations and Misuses of LSRL★★★★☆⏱ 3 min

LSRL only describes linear relationships. If the underlying bivariate relationship is curved, the LSRL will produce biased predictions even within the observed x range. High-leverage outliers can also pull the LSRL line far away from the bulk of the data points.

📐 Worked Example

A LSRL predicting plant height from weeks of growth for 2 to 10 week old plants gives $\hat{h e i g h t} = 2 + 1.5 (w ee k s)$ . Explain why predicting height at 30 weeks is invalid.

1
The observed x range only covers 2 to 10 weeks of growth.
2
Plant growth slows and stops after maturity, so the linear trend cannot continue indefinitely.
3
Predicting height at 30 weeks is extrapolation, and will produce an unrealistic predicted height of 47 inches that does not match real plant behavior.

5. Common Pitfalls

Wrong move:

Interpreting the y-intercept even when x=0 is impossible or outside the observed data range

Why:

AP rubrics deduct points for ignoring if x=0 is a plausible value for the dataset

Correct move:

Explicitly note if x=0 is not a valid input before interpreting the intercept, or state the intercept has no practical meaning in context

Wrong move:

Calculating residual as predicted ŷ minus observed y

Why:

This reverses the sign of the residual, leading to incorrect statements about points lying above or below the line

Correct move:

Always use the formula residual = observed y - predicted ŷ for all AP exam calculations

Wrong move:

Stating the slope describes a causal relationship between x and y

Why:

Regression only describes associated trends, it cannot prove causation even for very strong linear fits

Correct move:

Frame all slope interpretations as predicted associated changes, not causal effects

Wrong move:

Omitting units from slope and intercept interpretations

Why:

AP scoring guidelines explicitly require units for full credit on interpretation prompts

Correct move:

Include full units for both the explanatory and response variables in every coefficient interpretation

Wrong move:

Using LSRL for datasets with clear non-linear curved trends

Why:

A linear model will systematically underpredict and overpredict values across the x range

Correct move:

Check residual plots first to confirm no curved pattern exists before using a linear LSRL

6. Quick Reference Cheatsheet

Quantity	Formula	AP Exam Required Interpretation
Slope $b_{1}$	$b_{1} = r \cdot \frac{s _{y}}{s _{x}}$	Predicted change in y for every 1 unit increase in x
Y-intercept $b_{0}$	$b_{0} = \overset{y}{ˉ} - b_{1} \overset{x}{ˉ}$	Predicted y value when x equals 0
Residual $e$	$e = y - \overset{y}{^}$	Difference between observed y and model predicted y
LSRL Equation	$\overset{y}{^} = b_{0} + b_{1} x$	Linear model for predicting response variable y from explanatory x

When this came up on past exams

AI-estimated based on syllabus patterns — cross-check with official past papers for accuracy. Use only as revision-focus signals.

2025 · Paper 1
Free response LSRL interpretation task
2024 · Paper 2
Multiple choice residual calculation
2023 · Paper 1
LSRL property verification question

What's Next

Mastering least-squares regression is the foundation for all subsequent inference for linear models on the AP Stats exam, which makes up roughly 15-20% of the total exam score. You will next build on this knowledge to analyze residual plots to assess if a linear model is appropriate for your dataset, calculate the coefficient of determination R² to quantify the proportion of variation explained by the model, and run hypothesis tests for the significance of the regression slope. These connected topics are frequently tested together in extended 10-point free response questions that require you to justify model selection, interpret outputs, and identify potential flaws in study design that would invalidate regression results. Prioritize practicing full free response prompts that combine LSRL interpretation with residual analysis to maximize your exam score.