Why is $X^T X$ a symmetric matrix?

Question

Using the notation that $X^T$ is the transpose of the matrix $X$, what is the 'deepest' way to show that $X^T X$ must be symmetric? (by deepest, I mean by quoting as few other results as possible)

I know that a symmetric matrix is a matrix for which $X$ = $X^T$, so applying that to the above:

$$(X^T X) = (X^T X)^T$$

for the matrix to be symmetric.

But I'm unsure of how to proceed from here.

Answer 1

You're forgetting that transposition is contravariant over multiplication:

$$(AB)^T=B^TA^T$$

Therefore

$$(X^TX)^T=X^T(X^T)^T$$

Transposition is also an involution, so

$$(X^T)^T=\cdots$$

Answer 2

You can argue using only the definition of matrix multiplication. Let $x_{ij}$ denote the entry in row $i$ and column $j$ of the matrix $X$ and let $y_{ij}$ denote the entry in row $i$ and column $j$ of the matrix $Y$.

As long as $Y$ is $m \times n$ and $X$ is $n \times p$ the product $Z=XY$ is $m \times p$ and $$z_{ij} = \sum_{k=1}^n y_{ik}x_{kj}.$$

If $Y = X^T$ then $y_{ij} = x_{ji}$ so in this case $$z_{ij} = \sum_{k=1}^n x_{ki} x_{kj}.$$ Here $i$ and $j$ are interchangable because $x_{ki} x_{kj} = x_{kj}x_{ki}$ so that $z_{ij} = z_{ji}$.