# Transition matrices and Markov chains

> IB Mathematics Applications and Interpretation HL · IB Math AI HL
> Source: https://www.owlsprep.com/study/ib-math-ai-hl-u5-transition-matrices-and-markov-chains/

We cover transition matrix construction, k-step state distribution calculations, regular Markov chain identification, and steady state solving for real-world stochastic modelling.

**Prerequisites:** [Square matrix multiplication and exponentiation](https://www.owlsprep.com/study/ib-math-ai-hl-u4-matrix-operations/); [Conditional and independent probability rules](https://www.owlsprep.com/study/ib-math-ai-hl-u5-independent-probability/)

## Learning objectives

- Construct valid n x n transition matrices for finite state Markov processes following IB convention
- Calculate state distribution vectors after k time steps using matrix exponentiation and multiplication
- Identify regular Markov chains and derive their unique steady state equilibrium distributions
- Apply Markov chain models to solve real-world prediction problems tested in IB Paper 2 and 3

## Constructing Valid Transition Matrices

All transition matrices for IB AI HL follow a strict column-stochastic convention: every entry is a non-negative probability, and the sum of values in each column equals exactly 1. Columns represent the current state at time t, and rows represent the next state at time t+1.

**Transition Matrix** — n x n square matrix for an n-state Markov process, where entry P_{i,j} gives the probability of moving from state j to state i in one time step.

*Notation:* P

**Worked example:** A 2-state model for gym attendance has states 'Attended' and 'Missed'. If a student attended this week, there is a 0.8 chance they attend next week, 0.2 chance they miss. If they missed this week, there is a 0.5 chance they attend next week, 0.5 chance they miss. Construct the valid transition matrix P.

1. Order states as [Attended, Missed] for both columns and rows
2. $$P = \begin{bmatrix} P(Attended \to Attended) & P(Missed \to Attended) \\ P(Attended \to Missed) & P(Missed \to Missed) \end{bmatrix}$$
3. Substitute the given probability values
4. $$P = \begin{bmatrix} 0.8 & 0.5 \\ 0.2 & 0.5 \end{bmatrix}$$
5. Validate column sums: 0.8 + 0.2 = 1, 0.5 + 0.5 = 1, confirming no errors

> **tip**
>
> Never use row-stochastic convention for IB AI HL, as all mark schemes use column sums equal to 1 for transition matrices.

**Check your understanding**

Test your understanding of valid transition matrix properties

1. Which of the following is a valid 2x2 IB convention transition matrix?

   - [[0.6, 0.3],[0.4, 0.7]]
   - [[0.5, 0.5],[0.5, 0.5]]
   - [[1.1, 0],[0, -0.1]]

   *Why:* All entries are non-negative, and each column sums exactly to 1.

## Calculating State Distributions After k Steps

The initial state distribution vector S_0 is a column vector listing the probability of being in each state at time t=0. For 1 step, the updated state is S_1 = P S_0. This recursive relation extends to k steps via matrix exponentiation.

**Derivation:** Prove the k-step state identity S_k = P^k S_0

*Starting from:* S_1 = P S_0

1. For t=2, substitute S_1 to get S_2 = P S_1 = P(P S_0) = P^2 S_0
2. Use induction: assume S_n = P^n S_0 holds for some integer n ≥ 1
3. Then S_{n+1} = P S_n = P(P^n S_0) = P^{n+1} S_0

*Conclusion:* The identity S_k = P^k S_0 holds for all positive integers k

**Worked example:** Using the gym attendance transition matrix P, if the student attended this week (S_0 = [1, 0]^T), find the probability they attend 3 weeks from now.

1. Write the initial state vector
2. $$S_0 = \begin{bmatrix} 1 \\ 0 \end{bmatrix}$$
3. Calculate P^3 using GDC matrix exponentiation
4. $$P^3 = \begin{bmatrix} 0.722 & 0.695 \\ 0.278 & 0.305 \end{bmatrix}$$
5. Multiply P^3 by S_0 to get S_3
6. $$S_3 = P^3 S_0 = \begin{bmatrix} 0.722 \\ 0.278 \end{bmatrix}$$
7. The probability of attending 3 weeks later is 0.722

## Regular Markov Chains and Steady State

All regular Markov chains converge to a unique steady state distribution as k approaches infinity, no matter what initial state vector you start with. This equilibrium is the long-term proportion of time the process spends in each state.

**Steady State Vector** — Non-negative column vector that satisfies P \pi = \pi, where the sum of all entries equals 1 to normalise total probability.

*Notation:* \pi

**Worked example:** Find the steady state vector for the gym attendance transition matrix P = [[0.8, 0.5],[0.2, 0.5]]

1. Define \pi = [a, b]^T, so P \pi = \pi, and a + b = 1
2. $$\begin{bmatrix} 0.8 & 0.5 \\ 0.2 & 0.5 \end{bmatrix} \begin{bmatrix}a \\b\end{bmatrix} = \begin{bmatrix}a \\b\end{bmatrix}$$
3. Expand the first equation: 0.8a + 0.5b = a → -0.2a + 0.5b = 0
4. Substitute b = 1 - a: -0.2a + 0.5(1 - a) = 0 → 0.5 = 0.7a
5. $$a = 5/7 ≈ 0.714, b = 2/7 ≈ 0.286$$
6. Long term, the student attends ~71.4% of weeks, and misses ~28.6% of weeks

**Exam command terms**

Common IB command terms for this topic and their requirements:

- **Show that the chain is regular** — Calculate P^2 or higher power and confirm no zero entries remain

- **Determine the long term proportion** — Solve for the steady state vector \pi using the normalisation constraint

## Markov Chain Modelling Conventions

Markov chains are a very common context for IB AI HL Paper 3 questions, where you will be asked to adapt a given model, run predictions, and justify if the steady state result is realistic for the given scenario.

> **mnemonic**
>
> Remember the convention: C = Column = Current state, R = Row = Result next state, to avoid mixing up transition matrix entries.

## Common pitfalls

- **Wrong:** Summing rows instead of columns to validate the transition matrix
  - Why it fails: Many general resources use row-stochastic convention, but IB AI HL strictly uses column-stochastic matrices
  - Correct: Always confirm every column adds exactly to 1 before performing any calculations
- **Wrong:** Using S_k = S_0 P^k (row vector multiplication)
  - Why it fails: This produces incorrect values for the IB column vector convention
  - Correct: Keep state vectors as columns, use the formula S_k = P^k S_0
- **Wrong:** Forgetting the sum of entries = 1 constraint when solving for steady state
  - Why it fails: The equation P \pi = \pi is linearly dependent, so it cannot be solved uniquely without the normalisation condition
  - Correct: Replace one of the linear equations derived from P \pi = \pi with the sum of probabilities equals 1 rule
- **Wrong:** Assuming all Markov chains have a unique steady state
  - Why it fails: Non-regular chains (absorbing states, periodic cycles) do not converge to a unique equilibrium
  - Correct: First confirm the chain is regular before attempting to calculate the steady state
- **Wrong:** Rounding intermediate values when calculating P^k for large k
  - Why it fails: Small rounding errors compound to produce drastically wrong final probabilities
  - Correct: Store full precision matrix values on your GDC and only round the final result

## Cheatsheet

| Quantity | Formula | Validation Rule |
| --- | --- | --- |
| Transition matrix P | P_{i,j} = P(j \to i) | All columns sum to 1, no negative entries |
| k-step state distribution | S_k = P^k S_0 | Sum of entries in S_k = 1 |
| Steady state \pi | P \pi = \pi, sum(\pi) = 1 | All entries of \pi are non-negative |
| Regular chain check | Find smallest k where P^k > 0 (all entries > 0) | No zero entries in P^k |

## What's next

Mastering Markov chains is a core requirement for scoring full marks on the 15-20 mark stochastic modelling questions that appear almost every year on IB AI HL Paper 3. You will next extend this knowledge to absorbing Markov chains, which are used to calculate expected number of steps before reaching an end state, and transition matrix applications for graph traversal. These skills also directly support your internal assessment if you choose to model a real-world dynamic system such as traffic flow, social media user engagement, or population migration.

---

From [OwlsPrep](https://www.owlsprep.com) — free study guides for A-Level, IB, AP and IGCSE, written against the official syllabus. Canonical page: https://www.owlsprep.com/study/ib-math-ai-hl-u5-transition-matrices-and-markov-chains/