# Least-Squares Regression

> AP Statistics · AP Statistics 2024-2026
> Source: https://www.owlsprep.com/study/ap-statistics-u14-least-squares-regression/

This module breaks down the least-squares regression line, the most widely used linear model for bivariate quantitative data on the AP Stats exam, including calculation, interpretation, residual analysis, and core limitations.

**Prerequisites:** [Understanding correlation coefficients and bivariate scatter plots](https://www.owlsprep.com/study/ap-statistics-u14-correlation-coefficients/); [Basic descriptive statistics for two-variable datasets](https://www.owlsprep.com/study/ap-statistics-u13-bivariate-descriptive-analysis/)

## Learning objectives

- Define the least-squares regression line (LSRL) and its core optimization criterion
- Calculate predicted values, residuals, and interpret slope/y-intercept in full context
- Verify that the LSRL minimizes the sum of squared residuals for a dataset
- Identify limitations of LSRL including extrapolation risks and non-linear data mismatches

## Core Definition of the Least-Squares Criterion

Unlike arbitrary manually fitted lines, the least-squares regression line is mathematically optimized to minimize the sum of squared vertical distances between each observed data point and the line. Squaring residuals penalizes larger deviations more heavily, preventing extreme outliers from skewing the model disproportionately.

**Least-Squares Regression Line** — The unique linear model that minimizes the value of $\sum (y_i - \hat{y}_i)^2$, the total sum of squared residuals across all data points.

*Notation:* \hat{y} = b_0 + b_1 x

**Worked example:** Given summary statistics for study time (x, minutes) vs exam score (y, points): $\bar{x}=45$, $s_x=12$, $\bar{y}=78$, $s_y=15$, $r=0.72$, calculate the full LSRL equation.

1. First calculate the slope $b_1$ using the correlation formula:
2. $$b_1 = r \cdot \frac{s_y}{s_x} = 0.72 \cdot \frac{15}{12} = 0.9$$
3. Next calculate the y-intercept $b_0$ using the point of means:
4. $$b_0 = \bar{y} - b_1 \bar{x} = 78 - (0.9 \cdot 45) = 78 - 40.5 = 37.5$$
5. Final LSRL equation: $\hat{y} = 37.5 + 0.9x$

**Check your understanding**

Test your understanding of the least-squares criterion:

1. What quantity does the LSRL minimize?

   - Sum of residuals
   - Sum of squared residuals
   - Sum of absolute deviations
   - Correlation coefficient

   *Why:* The least-squares criterion explicitly minimizes the total of squared vertical deviations between observed and predicted values.

> **Exam tip:** AP graders will deduct points if you define LSRL as minimizing "distance from points to the line" — you must specify vertical, squared deviations.

## Interpreting Coefficients and Residuals

Every coefficient interpretation on the AP exam must include full context of the variables, units, and explicit reference to predicted values rather than causal effects. Residuals tell you how far individual observed points lie above or below the model's prediction.

**Exam command terms**

These are the most common command terms used for LSRL questions on the AP exam:

- **Interpret the slope** — You must state the predicted change in y for a 1-unit increase in x, with full units for both variables

- **Calculate the residual** — You must show your work for observed y minus predicted ŷ, not just the final number

**Worked example:** For the LSRL $\hat{y} = 37.5 + 0.9x$ predicting exam score from study time, interpret the slope, then calculate the residual for a student who studied 50 minutes and scored 82 points.

1. Slope interpretation: For every additional 1 minute of study time, the predicted exam score increases by 0.9 points, on average.
2. Calculate predicted score for x=50:
3. $$\hat{y} = 37.5 + 0.9(50) = 82.5$$
4. Calculate residual as observed y minus predicted ŷ:
5. $$e = 82 - 82.5 = -0.5$$
6. Interpret residual: This student scored 0.5 points lower than the model predicted for their study time.

## Key Mathematical Properties of LSRL

- The LSRL always passes through the point of means $(\bar{x}, \bar{y})$
- The sum of all residuals for the full dataset equals exactly 0
- The correlation coefficient r describes how tightly points cluster around the LSRL
- The slope of the LSRL has the same sign as the correlation coefficient r

| Line Type | Sum of Squared Residuals | Passes Through $(\bar{x}, \bar{y})$ |
| --- | --- | --- |
| Arbitrary manually fitted line | Larger than LSRL value | Not guaranteed |
| LSRL | Minimum possible value | Guaranteed |

**Worked example:** Verify that the sum of residuals equals 0 for 3 data points: (30, 65), (45,78), (60,92) using the LSRL $\hat{y}=37.5+0.9x$.

1. Calculate predicted values for each x: $\hat{y}_1=64.5$, $\hat{y}_2=78$, $\hat{y}_3=91.5$
2. Calculate residuals: $e_1=65-64.5=0.5$, $e_2=78-78=0$, $e_3=92-91.5=0.5$
3. Sum residuals: $0.5 + 0 + (-0.5) = 0$

## Limitations and Misuses of LSRL

> **Extrapolation Risk**
>
> Extrapolation is one of the most commonly tested flaws in AP Stats free response questions. You must explicitly state that predictions outside the observed x range are unreliable, even if the linear fit is very strong.

LSRL only describes linear relationships. If the underlying bivariate relationship is curved, the LSRL will produce biased predictions even within the observed x range. High-leverage outliers can also pull the LSRL line far away from the bulk of the data points.

**Worked example:** A LSRL predicting plant height from weeks of growth for 2 to 10 week old plants gives $\hat{height} = 2 + 1.5(weeks)$. Explain why predicting height at 30 weeks is invalid.

1. The observed x range only covers 2 to 10 weeks of growth.
2. Plant growth slows and stops after maturity, so the linear trend cannot continue indefinitely.
3. Predicting height at 30 weeks is extrapolation, and will produce an unrealistic predicted height of 47 inches that does not match real plant behavior.

## Common pitfalls

- **Wrong:** Interpreting the y-intercept even when x=0 is impossible or outside the observed data range
  - Why it fails: AP rubrics deduct points for ignoring if x=0 is a plausible value for the dataset
  - Correct: Explicitly note if x=0 is not a valid input before interpreting the intercept, or state the intercept has no practical meaning in context
- **Wrong:** Calculating residual as predicted ŷ minus observed y
  - Why it fails: This reverses the sign of the residual, leading to incorrect statements about points lying above or below the line
  - Correct: Always use the formula residual = observed y - predicted ŷ for all AP exam calculations
- **Wrong:** Stating the slope describes a causal relationship between x and y
  - Why it fails: Regression only describes associated trends, it cannot prove causation even for very strong linear fits
  - Correct: Frame all slope interpretations as predicted associated changes, not causal effects
- **Wrong:** Omitting units from slope and intercept interpretations
  - Why it fails: AP scoring guidelines explicitly require units for full credit on interpretation prompts
  - Correct: Include full units for both the explanatory and response variables in every coefficient interpretation
- **Wrong:** Using LSRL for datasets with clear non-linear curved trends
  - Why it fails: A linear model will systematically underpredict and overpredict values across the x range
  - Correct: Check residual plots first to confirm no curved pattern exists before using a linear LSRL

## Cheatsheet

| Quantity | Formula | AP Exam Required Interpretation |
| --- | --- | --- |
| Slope $b_1$ | $b_1 = r \cdot \frac{s_y}{s_x}$ | Predicted change in y for every 1 unit increase in x |
| Y-intercept $b_0$ | $b_0 = \bar{y} - b_1 \bar{x}$ | Predicted y value when x equals 0 |
| Residual $e$ | $e = y - \hat{y}$ | Difference between observed y and model predicted y |
| LSRL Equation | $\hat{y} = b_0 + b_1 x$ | Linear model for predicting response variable y from explanatory x |

## What's next

Mastering least-squares regression is the foundation for all subsequent inference for linear models on the AP Stats exam, which makes up roughly 15-20% of the total exam score. You will next build on this knowledge to analyze residual plots to assess if a linear model is appropriate for your dataset, calculate the coefficient of determination R² to quantify the proportion of variation explained by the model, and run hypothesis tests for the significance of the regression slope. These connected topics are frequently tested together in extended 10-point free response questions that require you to justify model selection, interpret outputs, and identify potential flaws in study design that would invalidate regression results. Prioritize practicing full free response prompts that combine LSRL interpretation with residual analysis to maximize your exam score.

---

From [OwlsPrep](https://www.owlsprep.com) — free study guides for A-Level, IB, AP and IGCSE, written against the official syllabus. Canonical page: https://www.owlsprep.com/study/ap-statistics-u14-least-squares-regression/