Science:Math Exam Resources/Courses/MATH307/April 2009/Question 08 (a)/Solution 1

Since the matrix is stochastic, all columns should sum to 1. We see that we have a 1/3 chance of moving to square 1 and a 2/3 chance of moving to square 3 from square 2, since the probability of rolling any specific number on the die is 1/6 and the probabilities are independent, so $P(AorB)=P(A)+P(B)$; similarly, we have a 1/3 chance of moving to square 2 and a 2/3 chance of moving to square 4 from square 3. If we get to squares 1 or 4, then we stay there, since the game is over. So let column 1 represent the first square of the game, column 2 the second, column 3 the third, and column 4 the fourth. Then the stochastic matrix representing the behaviour of the game is

$\left[\begin{array}{cccc}1& \frac{1}{3}& 0& 0\\ 0& 0& \frac{1}{3}& 0\\ 0& \frac{2}{3}& 0& 0\\ 0& 0& \frac{2}{3}& 1\end{array}\right]$