Transition matrices and Markov chains
IB Mathematics Applications and Interpretation HLΒ· 12 min read
1. Constructing Valid Transition Matricesβ β ββββ± 3 min
All transition matrices for IB AI HL follow a strict column-stochastic convention: every entry is a non-negative probability, and the sum of values in each column equals exactly 1. Columns represent the current state at time t, and rows represent the next state at time t+1.
Transition Matrix
n x n square matrix for an n-state Markov process, where entry P_{i,j} gives the probability of moving from state j to state i in one time step.
A 2-state model for gym attendance has states 'Attended' and 'Missed'. If a student attended this week, there is a 0.8 chance they attend next week, 0.2 chance they miss. If they missed this week, there is a 0.5 chance they attend next week, 0.5 chance they miss. Construct the valid transition matrix P.
- 1
Order states as [Attended, Missed] for both columns and rows
- 2
- 3
Substitute the given probability values
- 4
- 5
Validate column sums: 0.8 + 0.2 = 1, 0.5 + 0.5 = 1, confirming no errors
Test your understanding of valid transition matrix properties
Which of the following is a valid 2x2 IB convention transition matrix?
[[0.6, 0.3],[0.4, 0.7]]
[[0.5, 0.5],[0.5, 0.5]]
[[1.1, 0],[0, -0.1]]
Reveal answer
[[0.5, 0.5],[0.5, 0.5]] βAll entries are non-negative, and each column sums exactly to 1.
2. Calculating State Distributions After k Stepsβ β β βββ± 3 min
The initial state distribution vector S_0 is a column vector listing the probability of being in each state at time t=0. For 1 step, the updated state is S_1 = P S_0. This recursive relation extends to k steps via matrix exponentiation.
Prove the k-step state identity S_k = P^k S_0
S_1 = P S_0
- 1
For t=2, substitute S_1 to get S_2 = P S_1 = P(P S_0) = P^2 S_0
- 2
Use induction: assume S_n = P^n S_0 holds for some integer n β₯ 1
- 3
Then S_{n+1} = P S_n = P(P^n S_0) = P^{n+1} S_0
The identity S_k = P^k S_0 holds for all positive integers k
Using the gym attendance transition matrix P, if the student attended this week (S_0 = [1, 0]^T), find the probability they attend 3 weeks from now.
- 1
Write the initial state vector
- 2
- 3
Calculate P^3 using GDC matrix exponentiation
- 4
- 5
Multiply P^3 by S_0 to get S_3
- 6
- 7
The probability of attending 3 weeks later is 0.722
3. Regular Markov Chains and Steady Stateβ β β β ββ± 4 min
All regular Markov chains converge to a unique steady state distribution as k approaches infinity, no matter what initial state vector you start with. This equilibrium is the long-term proportion of time the process spends in each state.
Steady State Vector
Non-negative column vector that satisfies P \pi = \pi, where the sum of all entries equals 1 to normalise total probability.
Find the steady state vector for the gym attendance transition matrix P = [[0.8, 0.5],[0.2, 0.5]]
- 1
Define \pi = [a, b]^T, so P \pi = \pi, and a + b = 1
- 2
- 3
Expand the first equation: 0.8a + 0.5b = a β -0.2a + 0.5b = 0
- 4
Substitute b = 1 - a: -0.2a + 0.5(1 - a) = 0 β 0.5 = 0.7a
- 5
- 6
Long term, the student attends ~71.4% of weeks, and misses ~28.6% of weeks
4. Markov Chain Modelling Conventionsβ β β βββ± 2 min
Markov chains are a very common context for IB AI HL Paper 3 questions, where you will be asked to adapt a given model, run predictions, and justify if the steady state result is realistic for the given scenario.
5. Common Pitfalls
Wrong move:
Summing rows instead of columns to validate the transition matrix
Why:
Many general resources use row-stochastic convention, but IB AI HL strictly uses column-stochastic matrices
Correct move:
Always confirm every column adds exactly to 1 before performing any calculations
Wrong move:
Using S_k = S_0 P^k (row vector multiplication)
Why:
This produces incorrect values for the IB column vector convention
Correct move:
Keep state vectors as columns, use the formula S_k = P^k S_0
Wrong move:
Forgetting the sum of entries = 1 constraint when solving for steady state
Why:
The equation P \pi = \pi is linearly dependent, so it cannot be solved uniquely without the normalisation condition
Correct move:
Replace one of the linear equations derived from P \pi = \pi with the sum of probabilities equals 1 rule
Wrong move:
Assuming all Markov chains have a unique steady state
Why:
Non-regular chains (absorbing states, periodic cycles) do not converge to a unique equilibrium
Correct move:
First confirm the chain is regular before attempting to calculate the steady state
Wrong move:
Rounding intermediate values when calculating P^k for large k
Why:
Small rounding errors compound to produce drastically wrong final probabilities
Correct move:
Store full precision matrix values on your GDC and only round the final result
6. Quick Reference Cheatsheet
Quantity | Formula | Validation Rule |
|---|---|---|
Transition matrix P | P_{i,j} = P(j \to i) | All columns sum to 1, no negative entries |
k-step state distribution | S_k = P^k S_0 | Sum of entries in S_k = 1 |
Steady state \pi | P \pi = \pi, sum(\pi) = 1 | All entries of \pi are non-negative |
Regular chain check | Find smallest k where P^k > 0 (all entries > 0) | No zero entries in P^k |
When this came up on past exams
AI-estimated based on syllabus patterns β cross-check with official past papers for accuracy. Use only as revision-focus signals.
- 2023 Β· Paper 2
Virus transmission Markov model
- 2022 Β· Paper 3
Retail customer loyalty simulation
- 2021 Β· Paper 2
Multi-state weather prediction
What's Next
Mastering Markov chains is a core requirement for scoring full marks on the 15-20 mark stochastic modelling questions that appear almost every year on IB AI HL Paper 3. You will next extend this knowledge to absorbing Markov chains, which are used to calculate expected number of steps before reaching an end state, and transition matrix applications for graph traversal. These skills also directly support your internal assessment if you choose to model a real-world dynamic system such as traffic flow, social media user engagement, or population migration.