I'm learning Linear Algebra on my own time. Currently studying projections. I understand all of the derivation, but there is one piece of it that bothers me.
Say I have a matrix $A$ that represents a plane in $3D$, (matrix has 2 columns and 3 rows), $b$ is a vector I'm trying to project onto a plane, $p$ is its projection and $e$ (error) is the vector normal to the plane, $e = b - p$
Given that $e$ is normal to the plane then it is normal to every vector in the plane, meaning the inner product with any vector in the plane is 0, i.e. $A^Te = 0$, substituting $e = b - p$ we get $$ A^T(b - p)= 0 $$ $$ A^Tb - A^Tp = 0 $$ $$ A^Tb = A^Tp $$ $$ b = p $$
My derivation clearly shows that projection equals to the vector being projected, but geometrically it is not so. What am I missing?
You claim that $(A^Tb = A^Tp) \Rightarrow (b=p)$. This isn't true.
E.g. let $A = \begin{pmatrix}1 & 0\\ 0 & 1\\ 0 & 0\end{pmatrix}$. This has two columns and three rows as required.
Let $b = \begin{pmatrix}1 \\ 1\\ 1\end{pmatrix}, c = \begin{pmatrix}1\\ 1\\ 2\end{pmatrix}$. Clearly $b \neq c$.
However, $A^Tb = \begin{pmatrix}1 & 0 & 0\\ 0 & 1 & 0\end{pmatrix}\begin{pmatrix}1 \\ 1\\ 1\end{pmatrix} = \begin{pmatrix}1 \\ 1\end{pmatrix} = \begin{pmatrix}1 & 0 & 0\\ 0 & 1 & 0\end{pmatrix}\begin{pmatrix}1 \\ 1\\ 2\end{pmatrix} = A^Tc$.
Intuitively this happens because $b, c$ are vectors in $\mathbb{R}^3$, but $A^Tb, A^Tc$ are vectors in $\mathbb{R}^2$, meaning you "lose information" in the projection.