I have a question regarding whether two vectors are orthogonal.
Say I have a vector say $x$, and I have a subspace say $A$ spanned by $a_1,a_2$, now say I project the vector $x$ onto the subspace of $A$, I know I can use the formula of $P=A(A^TA)^{-1}A$ where $P$ is the projection matrix to project onto $A$.
So say $x^*$ =$Px$, now let say I have a vector say $y$ living in the subspace $A$ that is orthogonal to $x^*$ (so $Ay=y$, since $y$ already living in the subspace $A$), so $y$ is orthogonal to the projected vector $x^*$.
my question is:
Is $y$ also being orthogonal to $x$ (i.e. the vector where $x^*$ was projected from)?
Say $x=x^*+x_1$ where $x^*$ is the orthogonal projection of $x$ onto $A$. Then $\langle x,y\rangle=\langle x^*,y\rangle+\langle x_1,y\rangle $. Now, $\langle x_1,y\rangle$ is zero as $x_1$ is orthogonal to anything in $A$ and the other term is zero because you imposed it to be so.