Science:Math Exam Resources/Courses/MATH152/April 2010/Question B 03 (c)

MATH152 April 2010

  •  QA 1  •  QA 2  •  QA 3  •  QA 4  •  QA 5  •  QA 6  •  QA 7  •  QA 8  •  QA 9  •  QA 10  •  QA 11  •  QA 12  •  QA 13  •  QA 14  •  QA 15  •  QA 16  •  QA 17  •  QA 18  •  QA 19  •  QA 20  •  QA 21  •  QA 22  •  QA 23  •  QA 24  •  QA 25  •  QA 26  •  QA 27  •  QA 28  •  QA 29  •  QA 30  •  QB 1(a)  •  QB 1(b)  •  QB 1(c)  •  QB 2(a)  •  QB 2(b)  •  QB 3(a)  •  QB 3(b)  •  QB 3(c)  •  QB 3(d)  •  QB 4(a)  •  QB 4(b)  •  QB 4(c)  •  QB 4(d)  •  QB 4(e)  •  QB 5(a)  •  QB 5(b)  •  QB 6(a)  •  QB 6(b)  •  QB 6(c)  •  QB 6(d)  •  QB 6(e)  •  

   Other MATH152 Exams

  •  April 2016  •  April 2015  •  April 2014  •  April 2022  •  April 2011  •  April 2010  •  April 2012  •  April 2013  •  April 2017  •  

Question B 03 (c)
A random walk problem with two states has a transition matrix $P=\left[{\begin{array}{cc}1/3&1/4\\2/3&3/4\end{array}}\right]$ c) Assume that the walker starts in state 1. Give an explicit formula that only depends on $n$ for the probability it is in state 1 after $n$ time steps.

Make sure you understand the problem fully: What is the question asking you to do? Are there specific conditions or constraints that you should take note of? How will you know if your answer is correct from your work only? Can you rephrase the question in your own words in a way that makes sense to you?

If you are stuck, check the hint below. Consider it for a while. Does it give you a new idea on how to approach the problem? If so, try it!

Hint
Use the information about the eigenvalues and eigenvectors of P to calculate the n-th power Pn of P.

Checking a solution serves two purposes: helping you if, after having used the hint, you still are stuck on the problem; or if you have solved the problem and would like to check your work. If you are stuck on a problem: Read the solution slowly and as soon as you feel you could finish the problem on your own, hide it and work on the problem. Come back later to the solution if you are stuck or if you want to check your work. If you want to check your work: Don't only focus on the answer, problems are mostly marked for the work you do, make sure you understand all the steps that were required to complete the problem and see if you made mistakes or forgot some aspects. Your goal is to check that your mental process was correct, not only the result.

Solution 1

We know that the probability vector after $n$ steps is determined by multiplying the initial state by the matrix $P$, $n$times. We begin by expressing the initial probability vector in terms of the eigenvectors of $P$. i.e: ${\begin{aligned}c_{1}\left[{\begin{array}{c}1/4\\2/3\end{array}}\right]+c_{2}\left[{\begin{array}{c}1\\-1\end{array}}\right]=\left[{\begin{array}{c}1\\0\end{array}}\right]\\\left[{\begin{array}{cc}1/4&1\\2/3&-1\end{array}}\right]\left[{\begin{array}{c}c_{1}\\c_{2}\end{array}}\right]=\left[{\begin{array}{c}1\\0\end{array}}\right]\\\left[{\begin{array}{cc}1/4&1\\11/12&0\end{array}}\right]\left[{\begin{array}{c}c_{1}\\c_{2}\end{array}}\right]=\left[{\begin{array}{c}1\\1\end{array}}\right]\\\left[{\begin{array}{cc}0&1\\1&0\end{array}}\right]\left[{\begin{array}{c}c_{1}\\c_{2}\end{array}}\right]=\left[{\begin{array}{c}8/11\\12/11\end{array}}\right]\end{aligned}}$ Thus $c_{1}=12/11$, $c_{2}=8/11$. Now that we know the values of the constants, we will exploit the relationship $P\mathbf {v} _{k}=\lambda _{k}\mathbf {v} _{k}\quad \rightarrow \quad P^{n}\mathbf {v} _{k}=\lambda _{k}^{n}\mathbf {v} _{k},$ which holds for eigenvectors vk. Using our answer from part b), we have the following ${\begin{aligned}P^{n}\left[{\begin{array}{c}1\\0\end{array}}\right]&=P^{n}(c_{1}\mathbf {v} _{1}+c_{2}\mathbf {v} _{2})\\&=c_{1}\lambda _{1}^{n}\mathbf {v} _{1}+c_{2}\lambda _{2}^{n}\mathbf {v} _{2}\\&={\frac {12}{11}}\left[{\begin{array}{c}1/4\\2/3\end{array}}\right]+{\frac {8}{11}}\left({\frac {1}{12}}\right)^{n}\left[{\begin{array}{c}1\\-1\end{array}}\right]\\&={\frac {1}{11}}\left[{\begin{array}{c}3+8\left({\frac {1}{12}}\right)^{n}\\8-8\left({\frac {1}{12}}\right)^{n}\end{array}}\right]\end{aligned}}$ Therefore, assuming that the walker starts in state 1, the probability that the walker is in state 1 after $n$ steps is ${\frac {1}{11}}\left(3+8\left({\frac {1}{12}}\right)^{n}\right)$

Solution 2

You can also solve this directly when you diagonalize P: $P=MDM^{-1}$ where M is the matrix with the eigenvectors of P as its columns, and D is a diagonal matrix with the corresponding eigenvalues on its diagonal. The catch is that then $P^{n}=MD^{n}M^{-1}$ and that powers of a diagonal matrix D are very easy to calculate. So let's do the work: Recall that and an eigenvector to the eigenvalue 1 is $v=\left[{\begin{array}{c}1/4\\2/3\end{array}}\right]$, and an eigenvector to the eigenvalue 1/12 is $w=\left[{\begin{array}{c}1\\-1\end{array}}\right]$. Possible choices of M and D are therefore ${\begin{aligned}M&=\left[{\begin{array}{cc}1/4&1\\2/3&-1\end{array}}\right]\quad {\text{and}}\quad D&=\left[{\begin{array}{cc}1&0\\0&1/12\end{array}}\right]\\{\text{or}}\\M&=\left[{\begin{array}{cc}1&1/4\\-1&2/3\end{array}}\right]\quad {\text{and}}\quad D&=\left[{\begin{array}{cc}1/12&0\\0&1\end{array}}\right]\\\end{aligned}}$ Multiplying any column of M (but not of D) by a non-zero constant is also possible. We choose the former matrices M and D. Next step is to find M-1, which is particularly easy for a 2x2 matrix since we use the formula $\left(\left[{\begin{array}{cc}a&b\\c&d\end{array}}\right]\right)^{-1}={\frac {1}{ad-bc}}\left[{\begin{array}{cc}d&-b\\-c&a\end{array}}\right].$ Therefore, we obtain ${\begin{aligned}P&=MDM^{-1}\\\left[{\begin{array}{cc}1/3&1/4\\2/3&3/4\end{array}}\right]&=\left[{\begin{array}{cc}1/4&1\\2/3&-1\end{array}}\right]\left[{\begin{array}{cc}1&0\\0&1/12\end{array}}\right]{\frac {-12}{11}}\left[{\begin{array}{cc}-1&-1\\-2/3&1/4\end{array}}\right]\end{aligned}}$ Therefore ${\begin{aligned}P^{n}&=MD^{n}M^{-1}\\&=\left[{\begin{array}{cc}1/4&1\\2/3&-1\end{array}}\right]\left[{\begin{array}{cc}1^{n}&0\\0&(1/12)^{n}\end{array}}\right]{\frac {-12}{11}}\left[{\begin{array}{cc}-1&-1\\-2/3&1/4\end{array}}\right]\\\\&={\frac {-12}{11}}\left[{\begin{array}{cc}-{\frac {1}{4}}-{\frac {2}{3}}({\frac {1}{12}})^{n}&-{\frac {1}{4}}+{\frac {1}{4}}({\frac {1}{12}})^{n}\\-{\frac {2}{3}}+{\frac {2}{3}}({\frac {1}{12}})^{n}&-{\frac {2}{3}}-{\frac {1}{4}}({\frac {1}{12}})^{n}\end{array}}\right]\\\\&={\frac {1}{11}}\left[{\begin{array}{cc}3+8({\frac {1}{12}})^{n}&3-3({\frac {1}{12}})^{n}\\8-8({\frac {1}{12}})^{n}&8+3({\frac {1}{12}})^{n}\end{array}}\right]\end{aligned}}$ Hence, finally, we can conclude that ${\begin{aligned}P^{n}\left[{\begin{array}{c}1\\0\end{array}}\right]&={\frac {1}{11}}\left[{\begin{array}{cc}3+8({\frac {1}{12}})^{n}&3-3({\frac {1}{12}})^{n}\\8-8({\frac {1}{12}})^{n}&8+3({\frac {1}{12}})^{n}\end{array}}\right]\left[{\begin{array}{c}1\\0\end{array}}\right]\\&={\frac {1}{11}}\left[{\begin{array}{c}3+8({\frac {1}{12}})^{n}\\8-8({\frac {1}{12}})^{n}\end{array}}\right]\end{aligned}}$ which means that the probability of being at stage 1 after n steps is ${\frac {1}{11}}\left(3+8\left({\frac {1}{12}}\right)^{n}\right)$