How eigendecomposition helps to solve $\operatorname*{argmax}_d\operatorname{Tr}(d^TX^TXd)$ in PCA

Question

I cannot understand the highlighted statement in the book Deep Learning. Can someone explain why we can solve this optimization problem using eigendecomposition?

Thank you

Answer 1

The key is that $X^TX$ is a symmetric matrix, and $d^T(X^TX)d$ is the Rayleigh quotient; see also "Rayleigh's theorem". For a reference on all that, I'd recommend either Horn and Johnson's Matrix Analysis or Bhatia's Matrix Analysis.