Find maximum of matrix trace

Question

I want to solve the following:

$\max_d{\operatorname{Tr}(d^TX^TXd)}$ subject to $d^Td=1$

I would assume that I can get the maximum vector $d$ by calculating the eigenvector of $X^TX$.

Is this the right way?

Answer 1

First, you can drop the trace since $d^TX^TXd$ is already a scalar. Without the trace, the problem you describe is the same as finding the first principal component. You want to $$ \underset{d}{\text{maximize}}\quad d^TX^TXd $$ subject to $$ d^Td=1 $$ You can rewrite this on the Lagrangian form $$ L(d,\lambda) = d^TX^TXd-\lambda(d^Td-1) $$ Differentiate $L$ with respect to $d$ yields $$ \frac{\partial L}{\partial d}=2(X^TX-\lambda)d=0 $$ or $$ X^TX d=\lambda d $$ Thus $$ d^TX^TXd = \lambda $$ which shows that the maximum value of your problem is the largest eigenvalue, and the $d$ that achieves this value is the eigenvector corresponding to the largest eigenvalue. So, yes you are correct in your assumption.