If the matrix of a linear transformation T$\colon \mathbb{R}^{N} \rightarrow \mathbb{R}^{N}$ with respect to some basis is symmetric, what does it say about the transformation? Is there a way to geometrically interpret the transformation in a nice/simple way?
I have tried to think about this and look it up on the web and in a few books, but I haven't found anything. It's just that the matrix being symmetric looks like such a nice property, there must be something special about the transformation too? Thanks for any help.
If $\mathbb{R}^n$ is endowed with an inner product $\langle\,\cdot\, , \,\cdot\,\rangle$ and the matrix $A$ of $T$ is symmetric with respect to an orthonormal basis, then we have the important property that $$\langle y, T(x) \rangle = \langle y, Ax \rangle = y Ax = y^TA^Tx = (Ay)^T x = \langle Ay , x\rangle = \langle T(y), x \rangle;$$ in this case we say that $T$ itself is symmetric (w.r.t. the inner product). There's too much to say about why these are important in a single post, but let me point out two useful facts:
(1) By the Spectral Theorem, $T$ is orthogonally diagonalizable, that is, $T$ is conjugate by an orthogonal transformation to a diagonal transformation.
(2) Suppose $x, y$ are eigenvectors of $T$. If they correspond respectively to distinct eigenvalues $\lambda, \mu$, then we have
$$\lambda \langle x, y \rangle = \langle \lambda x, y \rangle = \langle T(x), y \rangle = \langle x, T(y) \rangle = \langle x, \mu y \rangle = \mu \langle x, y \rangle.$$ In particular, if $\lambda \neq \mu$ then $\langle x, y \rangle = 0$, that is, the eigenspaces of $T$ are all orthogonal!
Finally, let me note that there is a comparably important complex version of this, in which we ask for a complex transformation w.r.t. a Hermitian inner product to be Hermitian, i.e., to ask $A^* = A$, where ${}^*$ denotes the conjugate transpose. We call such transformations self-adjoint, and they are of fundamental importance to, for example, quantum mechanics.