Complex Inner Product of Vectors

Question

In the case of vectors $a,b \in \mathbb{R}^n$ with inner product $\langle a,b\rangle=a^Tb$ we do not need to worry about whether we mean to project $a$ onto $b$ or $b$ onto $a$ because $\langle a,b\rangle=\langle b,a\rangle$.

However, for $a,b \in \mathbb{C}^n$ we now need to be careful about what we mean since $\langle a, b\rangle=a^Hb \neq \langle b, a\rangle$. If I want to project $a$ onto $b$, does that correspond to $\langle a,b\rangle$ or $\langle b,a\rangle$? How would I show this to myself?

Answer 1

(Your convention with the conjugate on the first vector is far from universal, by the way.) :)

Well, you want to write $a = cb + v$, where $c$ is a scalar and $v$ is orthogonal to $b$. So, taking the hermitian inner product with $b$, we have $$\langle b,a\rangle = \langle b,cb+v\rangle = c\|b\|^2 + \langle b,v\rangle,$$ so, not surprisingly, $$c = \frac{\langle b,a\rangle}{\|b\|^2}.$$ Note that if you did the inner product "on the other side," here's what you'll get: $$\langle a,b\rangle = \langle cb,b\rangle + \langle v,b\rangle.$$ The last term still vanishes, of course, but what about the first term on the right? It will become $\bar c\|b\|^2$, so now you have $$\bar c = \frac{\langle a,b\rangle}{\|b\|^2}.$$ The answers are in agreement.

Answer 2

This answer assumes that $\langle ,\rangle$ is linear in the first variable and antilinear in the second: $\langle \alpha x,y\rangle = \alpha \langle x,y\rangle$, but $\langle x,\alpha y\rangle = \overline{\alpha} \langle x,y\rangle$. This is the reverse of what you get if you define $\langle x,y\rangle = x^* y$. But whatever your convention is, you can use this logic to figure out the right order to use.

---

To project $a$ onto $b$, we want to write $a$ as a sum $a = u + v$ where $u = \alpha b$ for some scalar $\alpha$ and $\langle v,b\rangle = 0$. (The second condition is equivalent to asking that $\langle b,v\rangle = \overline{\langle v,b\rangle} = \overline{0} = 0$.) Then $u$ is the projection we wanted.

To derive the formula for a projection, we could either consider $\langle a,b\rangle = \langle u+v,b\rangle$ and simplify, or consider $\langle b,a\rangle = \langle b,u+v\rangle$ and simplify.

If we try the first one, then we have $$ \langle a,b\rangle = \langle u,b\rangle + \langle v,b\rangle = \langle \alpha b,b\rangle = \alpha \langle b,b\rangle $$ so we can solve for $\alpha$: $\alpha = \frac{\langle a,b\rangle}{\langle b,b\rangle}$ and therefore $u = \frac{\langle a,b\rangle}{\langle b,b\rangle}b$ is the projection of $a$ onto $b$.

If we try the second one, then we have $$ \langle b,a\rangle = \langle b,u\rangle + \langle b,v\rangle = \langle b, \alpha b\rangle = \overline{\alpha}\langle b,b\rangle $$ instead, because the complex inner product is not linear but anti-linear in the second coordinate. So we get $\overline{\alpha} = \frac{\langle b,a\rangle}{\langle b,b\rangle}$, and we need to take a complex conjugate to get $\alpha$.