How do I exactly project a vector onto a subspace?

Question

I am trying to understand how - exactly - I go about projecting a vector onto a subspace.

Now, I know enough about linear algebra to know about projections, dot products, spans, etc etc, so I am not sure if I am reading too much into this, or if this is something that I have missed.

For a class I am taking, the proff is saying that we take a vector, and 'simply project it onto a subspace', (where that subspace is formed from a set of orthogonal basis vectors).

Now, I know that a subspace is really, at the end of the day, just a set of vectors. (That satisfy properties here). I get that part - that its this set of vectors. So, how do I "project a vector on this subspace"?

Am I projecting my one vector, (lets call it a[n]) onto ALL the vectors in this subspace? (What if there is an infinite number of them?)

For further context, the proff was saying that lets say we found a set of basis vectors for a signal, (lets call them b[n] and c[n]) then we would project a[n] onto its signal subspace. We project a[n] onto the signal-subspace formed by b[n] and c[n]. Well, how is this done exactly?..

Thanks in advance, let me know if I can clarify anything!

P.S. I appreciate your help, and I would really like for the clarification to this problem to be somewhat 'concrete' - for example, something that I can show for myself over MATLAB. Analogues using 2-D or 3-D space so that I can visualize what is going on would be very much appreciated as well.

Thanks again.

Answer 1

I will talk about orthogonal projection here.

When one projects a vector, say $v$, onto a subspace, you find the vector in the subspace which is "closest" to $v$. The simplest case is of course if $v$ is already in the subspace, then the projection of $v$ onto the subspace is $v$ itself.

Now, the simplest kind of subspace is a one dimensional subspace, say the subspace is $U = \operatorname{span}(u)$. Given an arbitrary vector $v$ not in $U$, we can project it onto $U$ by $$v_{\| U} = \frac{\langle v , u \rangle}{\langle u , u \rangle} u$$ which will be a vector in $U$. There will be more vectors than $v$ that have the same projection onto $U$.

Now, let's assume $U = \operatorname{span}(u_1, u_2, \dots, u_k)$ and, since you said so in your question, assume that the $u_i$ are orthogonal. For a vector $v$, you can project $v$ onto $U$ by $$v_{\| U} = \sum_{i =1}^k \frac{\langle v, u_i\rangle}{\langle u_i, u_i \rangle} u_i = \frac{\langle v , u_1 \rangle}{\langle u_1 , u_1 \rangle} u_1 + \dots + \frac{\langle v , u_k \rangle}{\langle u_k , u_k \rangle} u_k.$$

Answer 2

Take a basis $\{v_1, \dots, v_n\}$ for the "signal subspace" $V$. Let's assume $V$ is finite dimensional for simplicity and practical purposes, but you can generalize to infinite dimensions. Let's also assume the basis is orthonormal.

The projection of your signal $f$ onto the subspace $V$ is just

$$\mathrm{proj}_V(f) = \sum_{i=1}^n \langle f, v_i \rangle v_i$$

and $f = \mathrm{proj}_V(f) + R(f)$, where $R(f)$ is the remainder, or orthogonal complement, which will be 0 if $f$ lies in the subspace $V$.

The $i$-th term of the sum, $\langle f, v_i\rangle$, is the projection of $f$ onto the subspace spanned by the $i$-th basis vector. (Note, if the $v_i$ are orthogonal, but not necessarily orthonormal, you must divide the $i$-th term by $\|v_i\|^2$.)