Generating Functions

CIE A-Level Further Mathematics· Unit 4: Further Probability & Statistics, Generating functions· 20 min read

1. PGFs: Definition and Core Properties★★☆☆☆⏱ 5 min

📘 Definition

Probability Generating Function

$G_{X} (t) = E (t^{X})$

For any non-negative integer-valued discrete random variable $X$ , the PGF is the expectation of $t^{X}$ , valid for all $t$ where the sum converges. Coefficients of $t^{x}$ equal $P (X = x)$ .

Example:

If $X \sim Bernoulli (p)$ , $G_{X} (t) = (1 - p) + pt$

We can find expectation and variance directly from PGF derivatives, avoiding summing over all possible values of $X$ :

G_X'(1) = \mathbb{E}(X) \\ G_X''(1) = \mathbb{E}[X(X-1)] = \mathbb{E}(X^2) - \mathbb{E}(X) \\ \text{Var}(X) = G_X''(1) + G_X'(1) - \left[G_X'(1)\right]^2

📐 Worked Example

A discrete random variable $X$ has PGF $G_{X} (t) = \frac{1}{2} t + \frac{1}{4} t^{2} + \frac{1}{4} t^{3}$ . Find $E (X)$ and $Var (X)$ .

1
Calculate the first derivative and evaluate at $t = 1$ :
2
$G_X'(t) = \frac{1}{2} + \frac{1}{2}t + \frac{3}{4}t^2 \\ G_X'(1) = \frac{1}{2} + \frac{1}{2} + \frac{3}{4} = \frac{7}{4} \\ \mathbb{E}(X) = \frac{7}{4} = 1.75$
3
Calculate the second derivative and evaluate at $t = 1$ :
4
$G_X''(t) = \frac{1}{2} + \frac{3}{2}t \\ G_X''(1) = \frac{1}{2} + \frac{3}{2} = 2$
5
Substitute into the variance formula:
6
$\text{Var}(X) = 2 + \frac{7}{4} - \left(\frac{7}{4}\right)^2 = \frac{32 + 28 - 49}{16} = \frac{11}{16} = 0.6875$

Exam tip:

Always evaluate PGF derivatives for moments at $t = 1$ , a common mistake is evaluating at $t = 0$ by accident.

2. PGFs for Common Discrete Distributions★★☆☆☆⏱ 6 min

You will be expected to recognize or derive PGFs for all standard discrete distributions tested in CIE 9231. The table below lists standard results:

Distribution	PMF	PGF
Binomial( $n, p$ )	$(x n) p^{x} (1 - p)^{n - x}$	$(1 - p + pt)^{n}$
Poisson( $λ$ )	$\frac{e ^{- λ} λ ^{x}}{x !}$	$e^{λ (t - 1)}$
Geometric( $p$ ) (trials to first success)	$(1 - p)^{x - 1} p$	$\frac{pt}{1 - ( 1 - p ) t}$
Negative Binomial( $r, p$ )	$(r - 1 x - 1) p^{r} (1 - p)^{x - r}$	$(\frac{pt}{1 - ( 1 - p ) t})^{r}$

📐 Worked Example

Derive the PGF of a Poisson distribution with parameter $λ$ , then find $E (X)$ from the PGF.

1
Start with the definition of PGF:
2
$G_X(t) = \mathbb{E}(t^X) = \sum_{x=0}^\infty t^x \cdot \frac{e^{-\lambda} \lambda^x}{x!}$
3
Factor out the constant term $e^{- λ}$ :
4
$G_X(t) = e^{-\lambda} \sum_{x=0}^\infty \frac{(\lambda t)^x}{x!}$
5
Use the Taylor series expansion $e^{y} = \sum_{x = 0}^{\infty} \frac{y ^{x}}{x !}$ :
6
$G_X(t) = e^{-\lambda} e^{\lambda t} = e^{\lambda(t-1)}$
7
Find the first derivative and evaluate at $t = 1$ :
8
$G_X'(t) = \lambda e^{\lambda(t-1)} \implies G_X'(1) = \lambda = \mathbb{E}(X)$

3. Sums of Independent Random Variables★★★☆☆⏱ 7 min

The most useful property of PGFs (and MGFs) is that the generating function of a sum of independent random variables is the product of the individual generating functions. This avoids the need for tedious convolution of probability mass functions.

📘 Definition

Product Property for Independent Sums

If $X_{1}, X_{2}, ..., X_{n}$ are independent discrete random variables, then the PGF of $S = X_{1} + X_{2} + ... + X_{n}$ is $G_{S} (t) = \prod_{i = 1}^{n} G_{X_{i}} (t)$ . The same rule applies to MGFs.

📐 Worked Example

Given independent $X \sim Poisson (λ)$ and $Y \sim Poisson (μ)$ , prove that $X + Y \sim Poisson (λ + μ)$ .

1
Write down the PGFs of $X$ and $Y$ :
2
$G_X(t) = e^{\lambda(t-1)}, \quad G_Y(t) = e^{\mu(t-1)}$
3
Use the product property for independent variables:
4
$G_{X+Y}(t) = G_X(t) G_Y(t) = e^{\lambda(t-1)} e^{\mu(t-1)} = e^{(\lambda+\mu)(t-1)}$
5
This PGF matches the PGF of a Poisson distribution with parameter $λ + μ$ . Since PGFs uniquely identify discrete distributions, $X + Y \sim Poisson (λ + μ)$ .

Exam tip:

Never use the product rule for dependent random variables. It only holds because independence gives $E (A B) = E (A) E (B)$ , which is not true for dependent variables.

4. Moment Generating Functions★★★☆☆⏱ 6 min

📘 Definition

Moment Generating Function

$M_{X} (t) = E (e^{tX})$

A generating function that works for both discrete and continuous random variables. The $n$ -th derivative evaluated at $t = 0$ gives the $n$ -th moment $E (X^{n})$ .

MGFs share the same product property as PGFs: for independent random variables, the MGF of the sum is the product of the individual MGFs. Like PGFs, MGFs uniquely identify probability distributions.

📐 Worked Example

Given $X \sim N (μ, σ^{2})$ , find the MGF of $X$ and use it to confirm $E (X) = μ$ .

1
Start with the definition of MGF for a continuous distribution:
2
$M_X(t) = \int_{-\infty}^\infty e^{tx} \cdot \frac{1}{\sigma \sqrt{2\pi}} e^{-(x-\mu)^2/(2\sigma^2)} dx$
3
Completing the square in the exponent simplifies the integral to give the standard result:
4
$M_X(t) = e^{\mu t + \frac{1}{2} \sigma^2 t^2}$
5
Differentiate once with respect to $t$ :
6
$M_X'(t) = (\mu + \sigma^2 t) e^{\mu t + \frac{1}{2} \sigma^2 t^2}$
7
Evaluate at $t = 0$ to get the expectation:
8
$M_X'(0) = (\mu + 0) e^0 = \mu = \mathbb{E}(X)$

5. Common Pitfalls

Wrong move:

Evaluating PGF derivatives at $t = 0$ instead of $t = 1$ for expectation

Why:

PGFs are defined as $G (t) = E (t^{X})$ , so derivatives only give moments at $t = 1$ . $t = 0$ only gives $P (X = 0)$ .

Correct move:

Always evaluate PGF derivatives for expectation/variance at $t = 1$ .

Wrong move:

Using the product rule for PGFs/MGFs of dependent random variables

Why:

The product rule relies on the property $E (A B) = E (A) E (B)$ , which only holds for independent variables.

Correct move:

Confirm the variables are independent before multiplying their generating functions.

Wrong move:

Omitting the $+ G^{'} (1)$ term in the variance formula for PGFs

Why:

$G^{''} (1) = E [X (X - 1)] = E (X^{2}) - E (X)$ , so $E (X)$ must be added back to get $E (X^{2})$ .

Correct move:

Use the full formula: $Var (X) = G^{''} (1) + G^{'} (1) - [G^{'} (1)]^{2}$ .

Wrong move:

Claiming two distributions are equal just because they have matching expectation and variance

Why:

Many different distributions share the same first two moments. Only matching PGFs/MGFs prove distributions are equal.

Correct move:

Always show the full PGF/MGF matches the target distribution to confirm identity.

Wrong move:

Using the wrong PGF for geometric distribution (wrong parameterization)

Why:

CIE uses the 'number of trials until first success' parameterization, not the 'number of failures before first success' version.

Correct move:

Memorize CIE's standard geometric PGF: $G (t) = \frac{pt}{1 - ( 1 - p ) t}$ .

6. Quick Reference Cheatsheet

Concept	Formula	Notes
PGF Definition	$G_{X} (t) = E (t^{X})$	Non-negative integer discrete $X$
$E (X)$ from PGF	$E (X) = G_{X}^{'} (1)$
Var $(X)$ from PGF	$G_{X}^{''} (1) + G_{X}^{'} (1) - [G_{X}^{'} (1)]^{2}$
PGF for independent sum	$G_{S} (t) = \prod G_{X_{i}} (t)$	Only for independent $X_{i}$
Binomial PGF	$(1 - p + pt)^{n}$
Poisson PGF	$e^{λ (t - 1)}$
Geometric PGF	$\frac{pt}{1 - ( 1 - p ) t}$	Trials to first success
MGF Definition	$M_{X} (t) = E (e^{tX})$	Discrete or continuous
$n$ -th moment from MGF	$E (X^{n}) = M_{X}^{(n)} (0)$
MGF for independent sum	$M_{S} (t) = \prod M_{X_{i}} (t)$	Only for independent $X_{i}$
Normal MGF	$e^{μ t + \frac{1}{2} σ^{2} t^{2}}$	$X \sim N (μ, σ^{2})$

When this came up on past exams

AI-estimated based on syllabus patterns — cross-check with official past papers for accuracy. Use only as revision-focus signals.

2024 · 2
Find expectation from given PGF
2023 · 2
PGF for sum of independent variables
2022 · 2
Derive MGF for binomial distribution
2021 · 2
Find variance of sum from PGFs

Going deeper

syllabusCIE 9231 Further Mathematics Syllabus

What's Next

Generating functions are a foundational tool for advanced probability and statistics, and underpin almost all further topics in university-level statistics. After mastering this sub-topic, you can apply generating function techniques to solve problems involving recurrence relations and random walks, a common follow-up topic in CIE 9231. Generating functions also form the basis for understanding sampling distributions, where properties of sums of independent random variables are core to hypothesis testing and confidence interval calculations. This topic connects directly to all further topics in statistics that involve combining information from multiple independent observations.

→Non-parametric tests