Minimizing the following objective function with matrices

Question

Suppose $A$ and $B$ are known matrices, and we are to find matrix $X$ that minimizes the following function,

$$\frac{1}{2}||X||^2+\frac{1}{2}||X^TAX-B||^2$$

Taking the relevant derivative w.r.t $X$ gives us, $$ X +(X^TAX-B)(A+A^T)X$$

I couldn't reduce the derivative further as $A$ is not necessarily a square matrix; so, $(A+A^T)$ would be invalid. I am using the formula in slide 17, Matrix Calculus to differentiate $X^TAX$ to $(A+A^T)X$

Is there a way to circumvent this issue?

Thank you.

Answer 1

By the AGM inequality, $\frac{1}{2}(||X||^2 + ||X^TAX - B||^2) \geq ||X||||X^TAX - B||$ with equality if $||X|| = ||X^TAX - B||$. So the minimum occurs when $X = X^TAX - B$.

Answer 2

Replace $X$ by $X+εH$ and disregard all terms that are $O(ε^2)$. Then, assuming the matrix norm is the Frobenius norm (?) \begin{align} &\tfrac12\|X+εH\|^2+\tfrac12\|(X+εH)^TA(X+εH)-B\|^2\\[1em] &=\tfrac12\|X\|^2+ε\,Tr(X^TH)\\[0.2em] &\qquad+\;\tfrac12\|X^TAX-B\|^2+ε\,Tr((X^TAX-B)^TH^TAX)+ε\,Tr((X^TAX-B)^TX^TAH)\\[1em] &=\tfrac12\|X\|^2+\tfrac12\|X^TAX-B\|^2\\[0.2em] &\qquad+ε\,Tr\Bigl(\Bigl[X^T+(X^TA^TX-B^T)X^TA+(X^TAX-B)X^TA^T\Bigr]H\Bigr) \end{align}

So the correct derivative is $$ X^T+(X^TA^TX-B^T)X^TA+(X^TAX-B)X^TA^T $$ or transposed as gradient $$ X+A^TX(X^TAX-B)+AX(X^TA^TX-B^T) $$ which has no easy further simplifications.