Let $T : V \to W$ be a linear transformation across inner product spaces over some field $\mathbb{F}$ and $T^* : W \to V$ be its adjoint. When considering $T$ composed with $T^*$, we have some unique properties:
- If $T^*T = TT^*$, $V = W$, and $\mathbb{F} = \mathbb{C}$, then $V$ has an orthonormal eigenbasis with respect to $T$.
- The eigenvalues of $T^*T$ are the singular values of $T$, which allows us to express $T$ as a diagonal matrix with respect to two orthonormal bases.
All of this seems pretty cool, but I don't really have any intuition for what $T^*T$ (or $TT^*$) means. Does anyone have an intuitive way of thinking about the transformation $T^*T$?
This transformation is called the Gramian matrix. It serves a multitude of roles (such as SVD/PCA). It is a "sort of squaring" for matrices (that can be rectangular, unlike the normal squaring). The best analogy for this "sort of squaring" I know of is in the complex numbers. Where $T^2$ is like $z^2$, you can think of $T^* T$ to be like $z \bar{z} = |z|^2$. For complex-valued matrices of size $1*1$, these two notions are even precisely equivalent. Both the (conjugate) transpose and conjugation are involutions; and this corresponds to the product of a value with its involution.
Also note, if your product is non-commutative (or only defined one way around), then $T^*T$ and $TT^*$ might not be equal.