I am trying to show that for a vector, $v$ in $R^n$ with $||v|| = 1$, that its projection onto it's span $\Omega = span(v)$ is given by $(v \cdot x)v$ for any $x \in R^n$.
I know how to do it through geometry, but I need to show it by solving directly for the point $z$ in $\Omega$ closest to $x: min_{z \in \Omega} ||x-z||^2$.
I've tried to do this by subbing in $cv$ for $z$ to make the expression: $||x - cv||^2$ and expanding it as a dot product to give me: $(x - cv) \cdot (x - cv) = x \cdot x - cv \cdot x - cv \cdot x + cv \cdot cv = ||x||^2 - 2c(v \cdot x) + c^2||v||^2$ but this is where I'm getting stuck and not seeing how to get from there to $(v \cdot x)v$.
Any insight would be greatly appreciated, and if possible, an extension of this to to have $v^{(1)}$ and $v^{(2)}$ be two orthonormal vectors and $\Omega = span(v^1, v^2)$ to have its projection be $(v^{(1)} \cdot x)v^{(1)} + (v^{(2)} \cdot x)v^{(2)}$.
Was able to find the answer with some help:
After taking the dot product and expanding, I ended up with $||x||^2 - 2c(v \cdot x) + c^2||v||^2$. I'm trying to optimize over c, so the first term can drop since it doesn't contain $c$, and since $||v|| = 1$, then $||v||^2 = 1$, so that term drops.
That left me with $- 2c(v \cdot x) + c^2$, which I treated as a function of c and differentiated it with respect to c to optimize: $\frac{d}{dc} - 2c(v \cdot x) + c^2 = - 2(v \cdot x) + 2c$.
Setting it to equal zero: $0 = - 2(v \cdot x) + 2c$ -> $-2c = - 2(v \cdot x)$ -> $c = (v \cdot x)$.
We can then plug the that back into $||x - cv||^2$ to get $||x - (v \cdot x)v||^2$