Science:Math Exam Resources/Courses/MATH221/April 2013/Question 11

{{#incat:MER QGQ flag|{{#incat:MER QGH flag|{{#incat:MER QGS flag|}}}}}}

MATH221 April 2013

  •  Q1  •  Q2 (a)  •  Q2 (b)  •  Q2 (c)  •  Q2 (d)  •  Q3  •  Q4  •  Q5  •  Q6  •  Q7 (a)  •  Q7 (b)  •  Q7 (c)  •  Q8  •  Q9  •  Q10  •  Q11  •  Q12 (a)  •  Q12 (b)  •  Q12 (c)  •  

   Other MATH221 Exams

  •  December 2008  •  December 2009  •  December 2007  •  April 2010  •  April 2009  •  December 2011  •  April 2013  •  

Question 11
Consider the follow matrix A and vector b: $A=\left[{\begin{array}{cc}1&-1\\1&0\\1&1\\1&2\end{array}}\right]\quad \mathbf {b} =\left[{\begin{array}{c}0\\1\\2\\2\end{array}}\right]$ Find the least squares solution x for the linear system Ax = b. Use this to find a linear function whose graph best fits the points (-1,0), (0,1), (1,2), and (2,2) in $\mathbb {R} ^{2}$.

Make sure you understand the problem fully: What is the question asking you to do? Are there specific conditions or constraints that you should take note of? How will you know if your answer is correct from your work only? Can you rephrase the question in your own words in a way that makes sense to you?

If you are stuck, check the hints below. Read the first one and consider it for a while. Does it give you a new idea on how to approach the problem? If so, try it! If after a while you are still stuck, go for the next hint.

Hint 1
What is the dimension of the least square solution vector x?

Hint 2
The equation Ax = b is overdetermined, and does not have a solution. What we mean by least square solution x is the vector ${\hat {\mathbf {x} }}$ that satisfies $\|A{\hat {\mathbf {x} }}-\mathbf {b} \|^{2}={\text{min}}_{\mathbf {x} \in \mathbb {R} ^{2}}\|A\mathbf {x} -\mathbf {b} \|^{2}$

Hint 3
In order to find the least square solution ${\hat {\mathbf {x} }}$ solve the normal equations $A^{T}A{\hat {\mathbf {x} }}=A^{T}\mathbf {b}$

Hint 4
Now that you know ${\hat {\mathbf {x} }}$ it is time to look at how the matrix A and the vector b were chosen. How does this relate to the equation of a line $\displaystyle y=mx+t$ Consider each of the four points, one at a time.

Checking a solution serves two purposes: helping you if, after having used all the hints, you still are stuck on the problem; or if you have solved the problem and would like to check your work. If you are stuck on a problem: Read the solution slowly and as soon as you feel you could finish the problem on your own, hide it and work on the problem. Come back later to the solution if you are stuck or if you want to check your work. If you want to check your work: Don't only focus on the answer, problems are mostly marked for the work you do, make sure you understand all the steps that were required to complete the problem and see if you made mistakes or forgot some aspects. Your goal is to check that your mental process was correct, not only the result.

Solution

The least squares solution vector ${\hat {\mathbf {x} }}$ is a 2x1 vector. We can quickly find it using the normal equations. ${\begin{aligned}A^{T}A{\hat {\mathbf {x} }}&=A^{T}\mathbf {b} \\\left[{\begin{array}{cccc}1&1&1&1\\-1&0&1&2\end{array}}\right]\left[{\begin{array}{cc}1&-1\\1&0\\1&1\\1&2\end{array}}\right]{\hat {\mathbf {x} }}&=\left[{\begin{array}{cccc}1&1&1&1\\-1&0&1&2\end{array}}\right]\mathbf {b} \\\left[{\begin{array}{cc}4&2\\2&6\end{array}}\right]{\hat {\mathbf {x} }}&=\left[{\begin{array}{c}5\\6\end{array}}\right]\end{aligned}}$ To find ${\hat {\mathbf {x} }}$ from here we can either invert the matrix on the left hand side, or use gaussian elimination. We choose the latter. ${\begin{aligned}\left[{\begin{array}{cc|c}4&2&5\\2&6&6\end{array}}\right]\rightarrow \left[{\begin{array}{cc|c}1&3&3\\4&2&5\end{array}}\right]\rightarrow \left[{\begin{array}{cc|c}1&3&3\\0&-10&-7\end{array}}\right]\rightarrow \left[{\begin{array}{cc|c}1&0&9/10\\0&1&7/10\end{array}}\right]\end{aligned}}$ Hence ${\hat {\mathbf {x} }}=\left[{\begin{array}{c}9/10\\7/10\end{array}}\right]$ Finally, how does this translate to the linear function whose graph best fits the four given points? In other words, find m and t such that $\displaystyle y=mx+t$ best fits the four given points (x1,y1), (x2,y2), (x3,y3), (x4,y4). To answer this we need to understand how the matrix A and the vector b were chosen. We notice that b holds the y values. Further, the x values of the points, are in the second column of A. The first column of A is ones and this is because the same constant t appears for each (x,y) pair. In matrix notation $\left[{\begin{array}{cc}1&x_{1}\\1&x_{2}\\1&x_{3}\\1&x_{4}\end{array}}\right]\left[{\begin{array}{cc}{\hat {\mathbf {x} }}_{1}\\{\hat {\mathbf {x} }}_{2}\end{array}}\right]=\left[{\begin{array}{cc}y_{1}\\y_{2}\\y_{3}\\y_{4}\end{array}}\right]$ Reading $\mathbf {b} =A{\hat {\mathbf {x} }}$ line by line reveals $y_{j}=1{\hat {\mathbf {x} }}_{1}+x_{j}{\hat {\mathbf {x} }}_{2}={\hat {\mathbf {x} }}_{2}x_{j}+{\hat {\mathbf {x} }}_{1}=7/10x_{j}+9/10$ for j = 1,2,3,4. Comparing to y = mx + t we obtain $m=7/10,\quad t=9/10$ The graph below is just for illustration purposes. It is not necessary to include a graph in your answer. Therefore, the linear function is given by $y={\frac {7}{10}}x+{\frac {9}{10}}$

{{#incat:MER CT flag||

Click here for similar questions

MER QGH flag, MER QGQ flag, MER QGS flag, MER QGT flag, MER Tag Least squares, Pages using DynamicPageList3 parser function, Pages using DynamicPageList3 parser tag

}}