SVD and projection

Question

I am working on part c of the problem above. I have figured out the expression for sum of square which is $ min \sum_{i} ||x_{i} - (x_{i}^{T} u)u||^{2}_{2}$, but could not work out how to use SVD for this.

Answer 1

Hint: Note that by the Pythagorean theorem and using the fact that $u$ is a unit vector, $$ \sum_i \|x_i - (x_i^Tu)u\|^2 = \sum_i (\|x_i\|^2 - \|(x_i^Tu)u\|^2)\\ = \sum_{i} \|x_i\|^2 - \sum_i |x_i^T u|^2\\ = \operatorname{tr}(X^TX) - \|Xu\|^2. $$ Use the SVD to find the unit vector $u$ that maximizes $\|Xu\|^2$ and thereby minimizes the cost function.