What is wrong with the following argument?
Let $H:\mathbb{R}^n\to\mathbb{R}^m$ be a linear map given by \begin{equation*} x=H(y)\quad\iff\quad\mathbf{x}=\mathbf{H}\mathbf{y}, \end{equation*} where $\mathbf{H}\in\mathbb{R}^{m\times n}$ is an $m$-by-$n$ real matrix. Then, why the following does not hold in general? \begin{align*} \mathbf{x}&=\mathbf{H}\mathbf{y}\\ \mathbf{H}^T\mathbf{x}&=\mathbf{H}^T\mathbf{H}\mathbf{y}\\ (\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T\mathbf{x}&=\mathbf{y}\\ \mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T\mathbf{x}&=\mathbf{H}\mathbf{y}\\ \mathbf{x}&=(\mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T)^{-1}\mathbf{H}\mathbf{y}, \end{align*} implying thereby that \begin{equation} \mathbf{I}_m=(\mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T)^{-1}, \end{equation} where $\mathbf{I}_m$ is the identity matrix in $\mathbb{R}^{m\times m}$.
Please note that:
- In the second line I multiplied both sides by $\mathbf{H}^T$.
- In the third line I multiplied both sides by $(\mathbf{H}^T\mathbf{H})^{-1}$.
- In the fourth line I multiplied both sides by $\mathbf{H}$.
- In the last line I multiplied both sides by $(\mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T)^{-1}$.
However, this conclusion seems not to be true. For instance, take: \begin{equation*} \mathbf{H}=\begin{bmatrix} 1 & 0 & 0\\ 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ 0 & 0 & 1\\ \end{bmatrix}\in\mathbb{R}^{5\times3}. \end{equation*} Then we get: \begin{equation*} (\mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T)^{-1}=\begin{bmatrix} 1/2 & 1/2 & 0 & 0 & 0\\ 1/2 & 1/2 & 0 & 0 & 0\\ 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1/2 & 1/2\\ 0 & 0 & 0 & 1/2 & 1/2\\ \end{bmatrix}^{-1}\neq\mathbf{I}_5. \end{equation*} In fact, it is easy to see that $\mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T$ is non-invertible.
To be honest, I am not sure where the argument went wrong. So if you have any ideas why I cannot manipulate the equation in such a way, please let me know.
Thank you, Richard.
You are assuming that $\mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T$ is invertible when you take the inverse. It certainly cannot be, for $\mathbf{H}$ has rank at most $3$. If $m\ne n$ considerations of rank show that either $\mathbf{H}^T\mathbf{H}$ is not invertible or $\mathbf{H}(\mathbf{H}^T\mathbf{H})^{-1}\mathbf{H}^T$ is not invertible.