Intuition for orthogonal vectors in $\Bbb R^n$

Question

Two vectors in $\Bbb R^n$ are orthogonal iff their dot product is $0$. I'm aware that the dot product can be defined in other spaces, but to keep things simple let's restrict ourselves to $\Bbb R^n$.

Given that the idea of orthogonality is roughly to identify when two vectors have no "overlap", then apart from the fact that in $\Bbb R^2$ and $\Bbb R^3$ this corresponds to the geometrical notion of orthogonality, why is this chosen as the definition of orthogonality? Ideally give examples of concrete mathematical problems where this definition arises naturally.

Answer 1

Orthoganility intuitively could be reflcted as "no dependency". It is deeply anchored in the intuition of eigen-space and eigen-vector and eigen-value, eigen-... Recall that eigen means in German "the very geniune property of itself" and this relates to a property which is absolutely genuine to $A$ and not shared with any $B$. So keeping also in mind that orthogonality of eigen-properties is relative. Indeed in $\Bbb R^n$ it ties depply with the dimmensions. There, dimmensions represent the number of possible eigen-properties (metaphoric: absolute selfish properties). The term orthogonality is lent indeed from the 90 degree or perpendicular $\perp$ visualisation of linear algebra but was by the mathematicians extended broadly to cover such type of relation that corresponds to eigen-properties under given constraints. From this point of view one should say orthogonal is a lent term that was used more and more expanded in its application to genuine independency and not restricted to be set identical to the term perpendicular.

Example: generally applicable you can test any $A$ and $B$ relatively to each other and under your given constraints with respect to orthogonality. A beautiful one might be that two theorems $T_1$ and $T_2$ could be orthogonal.

Answer 2

Firstly, let'd consider the geometry, to expand on Raskolnikov's comments:

Consider two independent vectors in $\mathbb R^n$. Then there's a unique plane through them (and the origin). Pick a linear transformation from $\mathbb R^n\to\mathbb R^2$ that sends that plane onto all of $\mathbb R^2$ and preserves lengths. Then the vectors are dot-product-orthogonal in $\mathbb R^n$ if and only if their images are geometrically-orthogonal in $\mathbb R^2$ (use the converse of Pythagorean theorem or whichever other criterion you like). This says that the geometry in $\mathbb R^n$ really acts the same way as in $\mathbb R^2$.

---

However, you seemed to be saying that you were not interested in the geometry side of things, and wanted an independent reason why orthogonality of vectors is important. Pretty much everything in your linear algebra book about orthogonality is an example, with one of the biggest ones being the spectral theorem, which says that every nice linear map (given symmetric matrices in the $\mathbb R^n$ case) is just a linear combination of orthogonal projections.

Another important use of orthogonality is in the singular value decomposition, which has many applications, including image compression and topography.