Suppose I have vector $x'=[1 $ $ $ $ x_2 $ $ $ $ x_3]$ and $x_3 = a + bx_2$ (where $a$ and $b$ are constant), and data, say $y$. In general, the least square will be $\beta = E[xx']^{-1}E[xy]$. Now, since $xx'$ is singular, how can I express the coefficient $\beta$?
From reading, I will need to find Moore-Penrose generalized inverse instead. However, I am not sure how this is done.
Thank you very much.
The Moore-Penrose pseudoinverse of a nonzero vector $v$ is $$ v^{+} = \frac{v^{*}}{v^{*} v}. $$ See Laub, example 4.8, p. 31.
To compute the Moore-Penrose pseudoinverse of a matrix $\mathbf{A}$, find the singular value decomposition: $$ \mathbf{A} = \mathbf{U} \left[ \begin{array}{cc} \mathbf{S} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] \mathbf{V}^{*} $$ The pseudoinverse is constructed using the prescription $$ \mathbf{A}^{+} = \mathbf{V} \left[ \begin{array}{cc} \mathbf{S}^{-1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{array} \right] \mathbf{U}^{*} $$