What does orthogonal mean in matrix theory?

Question

I know orthogonal involves right angles. However, in matrix theory,the concept still doesn't make sense to me. I was given a definition that vectors $x_1,...,x_k$ which belong to $\mathbb{C}^n$ form an orthogonal set if $x_i^* x_j=0$ for all pairs, $1\leq i<j\leq k$.

Answer 1

As it turns out: in $\Bbb C^n$, defining the orthogonality of vectors in terms of the angle between them is not exactly the "correct" thing to do.

To start, let's talk about the situation in $\Bbb R^n$. Note that the way we measure angles between two vectors $x,y \in \Bbb R^n$ is by looking at the dot-product $x^Ty$. More specifically, we can say that $$ x^Ty = \|x\| \cdot \|y\| \cdot \cos \theta. $$ If $x$ and $y$ are non-zero, then they will be perpendicular (i.e. orthogonal) iff (if and only if) $\theta = 90^\circ$, which happens iff $\cos \theta = 0$, which happens iff $x^Ty = 0$.

With that in mind, it is convenient to say that in $\Bbb R^n$, two vectors are defined to be orthogonal when their dot-product $x^Ty = 0$. The natural generalization of this definition is to say that two vectors in $\Bbb C^n$ are orthogonal if and only if their "dot-product" $x^*y$ is equal to zero.

Interestingly, it is no longer the case in $\Bbb C^n$ that non-zero vectors are orthogonal iff the angle between them is $90^\circ$.

Answer 2

This is really just the "nicest" way to extend the definition of orthogonal in real spaces to complex spaces. Note that in real vector spaces, $x\cdot y=\vert x\vert\vert y\vert \cos(\theta)$ where $\theta$ is the angle between the vectors $x$ and $y$. So as you say, if the angle between them is a right angle, then $\cos(\theta)=0$, so $x\cdot y=0$, and the vectors are orthogonal. Now also clearly for real vectors in a complex vector space, when we complex-conjugated the first vector, it doesn't change it, and so the same angle based orthogonality definition holds. But we can now also tell if a pair of complex vectors are "orthogonal", where it isn't so intuitively clear what orthogonal should mean (what is the angle between a pair of complex vectors?).

There are good reasons for conjugating the first argument, but I think the most intuitive one is that it means that the inner product of a vector with itself is always the square of its magnitude (because we know intuitively what the magnitude of a complex vector should be), so this property gets carried over from the dot product in real vector spaces.