Science:Math Exam Resources/Courses/MATH152/April 2011/Question A 22/Solution 1

If the transition matrix is

$P=\left[\begin{array}{ccc}8/10& 3/10& 1/10\\ 2/10& 3/10& 7/10\\ 0& 4/10& 2/10\end{array}\right]$

then the operation Px=b gives us the probability of being in each of the three states b based on where we started, x. We are told that we are equally likely to be in any of the three states so we have as our vector,

$\mathbf{𝐱}=[1/3,1/3,1/3].$

To find out the probabilities of being in each state after one transition we compute Px,

$P\mathbf{𝐱}=\left[\begin{array}{ccc}8/10& 3/10& 1/10\\ 2/10& 3/10& 7/10\\ 0& 4/10& 2/10\end{array}\right]\left[\begin{array}{c}1/3\\ 1/3\\ 1/3\end{array}\right]=\left[\begin{array}{c}2/5\\ 2/5\\ 1/5\end{array}\right].$

Therefore we get that the probability of being in state 3 is 1/5. Notice that the output probabilities sum to 1. This is a requirement since we only allow transitions from states 1 to 3, this summation to 1 simply says that no matter where we start we will end up somewhere in states 1 to 3.