From my understanding, orthogonal projection was finding parallel projection. However, in Gram-Schmidt, why are we looking for vectors that are perpendicular not parallel when we try to find orthogonal basis for a vector space? (like why is it subtracting v3 = u3 - proj(u3)) What does it mean by orthogonal in linear Algebra? What's the difference between orthogonal and orthogonal projection?
Thank you in advance!
What means parallel projection? It means you take a point , move it parallel to a given direction until it gets to a particular position, usually on a line. The orthogonal projection is a particular case of the above, when the direction that we want to project parallel to is perpendicular to the line that we want to project on.
Here is an example. Suppose you have the Cartesian coordinates in the plane, and a point at the vector position $(3,1)$. The orthogonal projection on the $x$ axis would be $3$. Now let's suppose you want to decompose the vector into two components, one along $x$ axis, and one along the diagonal in the first quadrant. Then, to project on the $x$ axis, you do a parallel projection, where the direction you project is parallel to $(1,1)$. So the answer will be $2$.
As for the last question, you need to have defined an inner product. Perpendicular means that the inner product is zero.