Science:Math Exam Resources/Courses/MATH152/April 2015/Question A 30/Solution 1

Let us examine what happens to the individual states

$${\mathbf {e_{1}}}={\begin{bmatrix}1\\0\\0\\0\end{bmatrix}},\quad {\mathbf {e_{2}}}={\begin{bmatrix}0\\1\\0\\0\end{bmatrix}},\quad {\mathbf {e_{3}}}={\begin{bmatrix}0\\0\\1\\0\end{bmatrix}},\quad {\mathbf {e_{4}}}={\begin{bmatrix}0\\0\\0\\1\end{bmatrix}}.$$

Examining ${\textstyle {\mathbf {e_{2}}}}$, we can see the 2nd column of ${\textstyle P}$ is

$${\begin{bmatrix}0\\1\\0\\0\end{bmatrix}}$$

so the second state is stationary (${\textstyle P^{n}{\mathbf {e_{2}}}={\mathbf {e_{2}}}}$ for all ${\textstyle n}$). Likewise, ${\textstyle P^{n}{\mathbf {e_{3}}}={\mathbf {e_{3}}}}$.

Examining ${\textstyle {\mathbf {e_{1}}}}$, we can see that every iteration this state has probability ${\textstyle 1/2}$ of becoming third state and ${\textstyle 1/2}$ of remaining the same (by the first column of ${\textstyle P}$). Since ${\textstyle {\mathbf {e_{3}}}}$ is stationary, this means that

$$P^{n}{\mathbf {e_{1}}}={\frac {1}{2^{n}}}{\mathbf {e_{1}}}+\left(1-{\frac {1}{2^{n}}}\right){\mathbf {e_{3}}}$$

, so as ${\textstyle n}$ goes to infinity, the probability of ${\textstyle {\mathbf {e_{1}}}}$ becoming the third state goes to one.

Similarly, as the number of iterations increases, the probability of ${\textstyle {\mathbf {e_{4}}}}$ becoming the second state goes to one.

Putting this all together, we have that the probability of ending up in the second state as ${\textstyle n\rightarrow \infty }$ is the sum of the probabilities of starting in the second and fourth states, i.e., ${\textstyle {\frac {7}{11}}}$, while by the same reasoning the probability of ending up in the third state is ${\textstyle {\frac {4}{11}}}$ (the remaining states have probability zero). This means our answer is

$$\lim _{n\rightarrow \infty }P^{n}{\mathbf {x_{0}}}=\color {blue}{\begin{bmatrix}0\\7/11\\4/11\\0\end{bmatrix}}.$$