Minimization of $\text{tr} (W^TMW)-\text{tr}(NW)$ subject to $W^TW=I$

Question

Is there a closed-form solution for finding W that minimizes the objective function: $\text{tr} (W^TMW)-\text{tr}(NW)$ subject to $W^TW=I$ where $M$ and $N$ are fixed matrices. I find it difficult to solve using Lagrange.

Answer 1

Assuming that $M$ is symmetric positive definite, then we can use the vec operator and the Kronecker product to translate the problem to a linear least squares problem over a sphere.

Let $\mathbf{w}=\mathrm{vec}(W)$ and $\mathbf{q}=\frac{1}{2}\mathbf{M}^{-1}\mathrm{vec}(N^T)$, where $\mathbf{M}=I\otimes M$ (the block diagonal matrix with $M$ repeated $k$-times on the diagonal with $k$ being the number of rows of $N$ and columns of $W$). Then $$ \mathrm{tr}(W^TMW) = \mathbf{w}^T\mathbf{M}\mathbf{w}, \qquad \mathrm{tr}(NW) = 2\mathbf{q}^T\mathbf{M}\mathbf{w}. $$ Note that $$ \|\mathbf{w}-\mathbf{q}\|_{\mathbf{M}}=(\mathbf{w}-\mathbf{q})^T\mathbf{M}(\mathbf{w}-\mathbf{q}) =\mathrm{tr}(W^TMW)-\mathrm{tr}(NW)+\mathbf{q}^T\mathbf{M}\mathbf{q}, $$ so minimizing $\|\mathbf{w}-\mathbf{q}\|_{\mathbf{M}}$ is equivalent to minimizing $\mathrm{tr}(W^TMW)-\mathrm{tr}(NW)$ since $\mathbf{q}^T\mathbf{M}\mathbf{q}$ is independent of $W$. The condition $W^TW=1$ translates simply to $\|\mathbf{w}\|_2=1$.

So the problem is equivalent to: find $\mathbf{w}$ such that $$ \|\mathbf{w}-\mathbf{q}\|_{\mathbf{M}}=\min_{\tilde{\mathbf{w}}}\|\tilde{\mathbf{w}}-\mathbf{q}\|_{\mathbf{M}}\quad\text{subject to}\quad\|\mathbf{w}\|_2=1. $$

This is essentially a sphere constrained least squares problem. You can find a solution method, e.g., in Matrix Computations by Golub and Van Loan.