# Generating Functions

> CIE A-Level Further Mathematics · Further Probability & Statistics Unit 4
> Source: https://www.owlsprep.com/study/cie-9231-u4-generating-functions/

This module covers probability and moment generating functions, core tools for finding moments and distributions of sums of independent random variables, frequently tested in CIE 9231 Paper 2.

**Prerequisites:** [Discrete random variables, expectation and variance](https://www.owlsprep.com/study/cie-9231-u4-discrete-random-variables/); Common discrete distributions (binomial, Poisson, geometric)

## Learning objectives

- Define probability generating functions (PGFs) and moment generating functions (MGFs) for random variables
- Calculate PGFs and MGFs for common discrete and continuous distributions
- Use PGFs/MGFs to find expectation, variance and higher moments
- Apply the product property to find the distribution of sums of independent random variables

## PGFs: Definition and Core Properties

**Probability Generating Function** — For any non-negative integer-valued discrete random variable $X$, the PGF is the expectation of $t^X$, valid for all $t$ where the sum converges. Coefficients of $t^x$ equal $P(X=x)$.

*Notation:* $G_X(t) = \mathbb{E}(t^X)$

*Example:* If $X \sim \text{Bernoulli}(p)$, $G_X(t) = (1-p) + pt$

We can find expectation and variance directly from PGF derivatives, avoiding summing over all possible values of $X$:

$$G_X'(1) = \mathbb{E}(X) \\
G_X''(1) = \mathbb{E}[X(X-1)] = \mathbb{E}(X^2) - \mathbb{E}(X) \\
\text{Var}(X) = G_X''(1) + G_X'(1) - \left[G_X'(1)\right]^2$$

**Worked example:** A discrete random variable $X$ has PGF $G_X(t) = \frac{1}{2}t + \frac{1}{4}t^2 + \frac{1}{4}t^3$. Find $\mathbb{E}(X)$ and $\text{Var}(X)$.

1. Calculate the first derivative and evaluate at $t=1$:
2. $$G_X'(t) = \frac{1}{2} + \frac{1}{2}t + \frac{3}{4}t^2 \\
G_X'(1) = \frac{1}{2} + \frac{1}{2} + \frac{3}{4} = \frac{7}{4} \\
\mathbb{E}(X) = \frac{7}{4} = 1.75$$
3. Calculate the second derivative and evaluate at $t=1$:
4. $$G_X''(t) = \frac{1}{2} + \frac{3}{2}t \\
G_X''(1) = \frac{1}{2} + \frac{3}{2} = 2$$
5. Substitute into the variance formula:
6. $$\text{Var}(X) = 2 + \frac{7}{4} - \left(\frac{7}{4}\right)^2 = \frac{32 + 28 - 49}{16} = \frac{11}{16} = 0.6875$$

> **Exam tip:** Always evaluate PGF derivatives for moments at $t=1$, a common mistake is evaluating at $t=0$ by accident.

## PGFs for Common Discrete Distributions

You will be expected to recognize or derive PGFs for all standard discrete distributions tested in CIE 9231. The table below lists standard results:

| Distribution | PMF | PGF |
| --- | --- | --- |
| Binomial($n,p$) | $\binom{n}{x} p^x (1-p)^{n-x}$ | $(1-p + pt)^n$ |
| Poisson($\lambda$) | $\frac{e^{-\lambda} \lambda^x}{x!}$ | $e^{\lambda(t-1)}$ |
| Geometric($p$) (trials to first success) | $(1-p)^{x-1}p$ | $\frac{pt}{1 - (1-p)t}$ |
| Negative Binomial($r,p$) | $\binom{x-1}{r-1} p^r (1-p)^{x-r}$ | $\left(\frac{pt}{1 - (1-p)t}\right)^r$ |

**Worked example:** Derive the PGF of a Poisson distribution with parameter $\lambda$, then find $\mathbb{E}(X)$ from the PGF.

1. Start with the definition of PGF:
2. $$G_X(t) = \mathbb{E}(t^X) = \sum_{x=0}^\infty t^x \cdot \frac{e^{-\lambda} \lambda^x}{x!}$$
3. Factor out the constant term $e^{-\lambda}$:
4. $$G_X(t) = e^{-\lambda} \sum_{x=0}^\infty \frac{(\lambda t)^x}{x!}$$
5. Use the Taylor series expansion $e^y = \sum_{x=0}^\infty \frac{y^x}{x!}$:
6. $$G_X(t) = e^{-\lambda} e^{\lambda t} = e^{\lambda(t-1)}$$
7. Find the first derivative and evaluate at $t=1$:
8. $$G_X'(t) = \lambda e^{\lambda(t-1)} \implies G_X'(1) = \lambda = \mathbb{E}(X)$$

## Sums of Independent Random Variables

The most useful property of PGFs (and MGFs) is that the generating function of a sum of independent random variables is the product of the individual generating functions. This avoids the need for tedious convolution of probability mass functions.

**Product Property for Independent Sums** — If $X_1, X_2, ..., X_n$ are independent discrete random variables, then the PGF of $S = X_1 + X_2 + ... + X_n$ is $G_S(t) = \prod_{i=1}^n G_{X_i}(t)$. The same rule applies to MGFs.

**Worked example:** Given independent $X \sim \text{Poisson}(\lambda)$ and $Y \sim \text{Poisson}(\mu)$, prove that $X+Y \sim \text{Poisson}(\lambda+\mu)$.

1. Write down the PGFs of $X$ and $Y$:
2. $$G_X(t) = e^{\lambda(t-1)}, \quad G_Y(t) = e^{\mu(t-1)}$$
3. Use the product property for independent variables:
4. $$G_{X+Y}(t) = G_X(t) G_Y(t) = e^{\lambda(t-1)} e^{\mu(t-1)} = e^{(\lambda+\mu)(t-1)}$$
5. This PGF matches the PGF of a Poisson distribution with parameter $\lambda+\mu$. Since PGFs uniquely identify discrete distributions, $X+Y \sim \text{Poisson}(\lambda+\mu)$.

> **Exam tip:** Never use the product rule for dependent random variables. It only holds because independence gives $\mathbb{E}(AB) = \mathbb{E}(A)\mathbb{E}(B)$, which is not true for dependent variables.

## Moment Generating Functions

**Moment Generating Function** — A generating function that works for both discrete and continuous random variables. The $n$-th derivative evaluated at $t=0$ gives the $n$-th moment $\mathbb{E}(X^n)$.

*Notation:* $M_X(t) = \mathbb{E}(e^{tX})$

MGFs share the same product property as PGFs: for independent random variables, the MGF of the sum is the product of the individual MGFs. Like PGFs, MGFs uniquely identify probability distributions.

**Worked example:** Given $X \sim N(\mu, \sigma^2)$, find the MGF of $X$ and use it to confirm $\mathbb{E}(X) = \mu$.

1. Start with the definition of MGF for a continuous distribution:
2. $$M_X(t) = \int_{-\infty}^\infty e^{tx} \cdot \frac{1}{\sigma \sqrt{2\pi}} e^{-(x-\mu)^2/(2\sigma^2)} dx$$
3. Completing the square in the exponent simplifies the integral to give the standard result:
4. $$M_X(t) = e^{\mu t + \frac{1}{2} \sigma^2 t^2}$$
5. Differentiate once with respect to $t$:
6. $$M_X'(t) = (\mu + \sigma^2 t) e^{\mu t + \frac{1}{2} \sigma^2 t^2}$$
7. Evaluate at $t=0$ to get the expectation:
8. $$M_X'(0) = (\mu + 0) e^0 = \mu = \mathbb{E}(X)$$

## Common pitfalls

- **Wrong:** Evaluating PGF derivatives at $t=0$ instead of $t=1$ for expectation
  - Why it fails: PGFs are defined as $G(t) = \mathbb{E}(t^X)$, so derivatives only give moments at $t=1$. $t=0$ only gives $P(X=0)$.
  - Correct: Always evaluate PGF derivatives for expectation/variance at $t=1$.
- **Wrong:** Using the product rule for PGFs/MGFs of dependent random variables
  - Why it fails: The product rule relies on the property $\mathbb{E}(AB) = \mathbb{E}(A)\mathbb{E}(B)$, which only holds for independent variables.
  - Correct: Confirm the variables are independent before multiplying their generating functions.
- **Wrong:** Omitting the $+ G'(1)$ term in the variance formula for PGFs
  - Why it fails: $G''(1) = \mathbb{E}[X(X-1)] = \mathbb{E}(X^2) - \mathbb{E}(X)$, so $\mathbb{E}(X)$ must be added back to get $\mathbb{E}(X^2)$.
  - Correct: Use the full formula: $\text{Var}(X) = G''(1) + G'(1) - [G'(1)]^2$.
- **Wrong:** Claiming two distributions are equal just because they have matching expectation and variance
  - Why it fails: Many different distributions share the same first two moments. Only matching PGFs/MGFs prove distributions are equal.
  - Correct: Always show the full PGF/MGF matches the target distribution to confirm identity.
- **Wrong:** Using the wrong PGF for geometric distribution (wrong parameterization)
  - Why it fails: CIE uses the 'number of trials until first success' parameterization, not the 'number of failures before first success' version.
  - Correct: Memorize CIE's standard geometric PGF: $G(t) = \frac{pt}{1 - (1-p)t}$.

## Cheatsheet

| Concept | Formula | Notes |
| --- | --- | --- |
| PGF Definition | $G_X(t) = \mathbb{E}(t^X)$ | Non-negative integer discrete $X$ |
| $\mathbb{E}(X)$ from PGF | $\mathbb{E}(X) = G_X'(1)$ |  |
| Var$(X)$ from PGF | $G_X''(1) + G_X'(1) - [G_X'(1)]^2$ |  |
| PGF for independent sum | $G_S(t) = \prod G_{X_i}(t)$ | Only for independent $X_i$ |
| Binomial PGF | $(1-p + pt)^n$ |  |
| Poisson PGF | $e^{\lambda(t-1)}$ |  |
| Geometric PGF | $\frac{pt}{1 - (1-p)t}$ | Trials to first success |
| MGF Definition | $M_X(t) = \mathbb{E}(e^{tX})$ | Discrete or continuous |
| $n$-th moment from MGF | $\mathbb{E}(X^n) = M_X^{(n)}(0)$ |  |
| MGF for independent sum | $M_S(t) = \prod M_{X_i}(t)$ | Only for independent $X_i$ |
| Normal MGF | $e^{\mu t + \frac{1}{2}\sigma^2 t^2}$ | $X \sim N(\mu, \sigma^2)$ |

## What's next

Generating functions are a foundational tool for advanced probability and statistics, and underpin almost all further topics in university-level statistics. After mastering this sub-topic, you can apply generating function techniques to solve problems involving recurrence relations and random walks, a common follow-up topic in CIE 9231. Generating functions also form the basis for understanding sampling distributions, where properties of sums of independent random variables are core to hypothesis testing and confidence interval calculations. This topic connects directly to all further topics in statistics that involve combining information from multiple independent observations.

- [Non-parametric tests](https://www.owlsprep.com/study/cie-9231-u4-non-parametric-tests/)

---

From [OwlsPrep](https://www.owlsprep.com) — free study guides for A-Level, IB, AP and IGCSE, written against the official syllabus. Canonical page: https://www.owlsprep.com/study/cie-9231-u4-generating-functions/