Maximize $\langle \mathrm A , \mathrm X \rangle$ subject to $\| \mathrm X \|_F = 1$

Question

Given $\mathrm A \in \mathbb R^{m \times n}$,

$$\begin{array}{ll} \text{maximize} & \langle \mathrm A , \mathrm X \rangle\\ \text{subject to} & \| \mathrm X \|_F = 1\end{array}$$

I can think of two approaches, which I write below. I am interested in other approaches.

---

#1

$$\langle \mathrm A , \mathrm X \rangle = \langle \mbox{vec} (\mathrm A) , \mbox{vec} (\mathrm X) \rangle \leq \|\mbox{vec} (\mathrm A)\|_2 \|\mbox{vec} (\mathrm X)\|_2 = \|\mathrm A\|_F \underbrace{\|\mathrm X\|_F}_{=1} = \|\mathrm A\|_F$$

Hence, the maximizer is $\mathrm X_{\max} := \gamma \mathrm A$, where $\gamma > 0$. From the constraint,

$$1 = \|\mathrm X_{\max}\|_F = \gamma \|\mathrm A\|_F$$

Thus,

$$\gamma = \dfrac{1}{\|\mathrm A\|_F}$$

and

$$\mathrm X_{\max} = \frac{\mathrm A \,\,\,}{\|\mathrm A\|_F}$$

---

#2

We define the Lagrangian

$$\mathcal{L} (\mathrm X, \lambda) := \langle \mathrm A , \mathrm X \rangle - \frac{\lambda}{2} (\| \mathrm X \|_F^2 - 1)$$

Taking the partial derivatives and finding where they vanish, we obtain

$$\mathrm A - \lambda \mathrm X = \mathrm O \qquad \qquad \qquad \| \mathrm X \|_F^2 = 1$$

Hence,

$$\mathrm X = \frac{1}{\lambda} \mathrm A$$

and

$$1 = \| \mathrm X \|_F^2 = \mbox{tr} (\mathrm X^T \mathrm X) = \frac{1}{\lambda^2} \mbox{tr} (\mathrm A^T \mathrm A) = \frac{1}{\lambda^2} \| \mathrm A \|_F^2$$

and, thus,

$$\lambda^2 = \| \mathrm A \|_F^2$$

or,

$$\lambda = \pm \| \mathrm A \|_F$$

Thus, the minimizer and maximizer are

$$\mathrm X_{\min} := -\frac{\mathrm A \,\,\,}{\| \mathrm A \|_F} \qquad \qquad \qquad \mathrm X_{\max} := \frac{\mathrm A \,\,\,}{\| \mathrm A \|_F}$$

and the minimum and maximum are $\pm \| \mathrm A \|_F$.

Answer 1

You are basically asking for alternative proofs of Cauchy-Schwarz inequality. There are plenty of them even on this site. A quick google search reveals an article that contains twelve proofs and I'd bet there is some literature out there that contains even more:

Hui-Hua Wu and Shanhe Wu, Various proofs of the Cauchy-Schwarz inequality.

Answer 2

Those are correct, but you can also use Cauchy-Schwarz inequality to deduce the answer.

Answer 3

This optimization problem is equivalent to maximizing

$$\text{vec}(A)^T\text{vec}(X)$$

over the $(mn-1)$-sphere. Thus the optimal point will just be the Frobenius normalization of $A$.

Answer 4

Make a change of variables to transform the problem into an unconstrained problem.
For typing convenience, I'll use a colon (:) to denote the matrix inner product, e.g. $$A:B={\rm tr}(A^TB)$$ Call the unconstrained variable $Y$, and find the differential of its length/norm. $$\eqalign{ \lambda^2 = \|Y\|_F^2 &= Y:Y \cr \lambda\,d\lambda &= Y:dY &\implies d\lambda &= \frac{Y:dY}{\lambda} \cr }$$ Replace $X$ by $Y/\lambda\,\,$ in the function and differentiate $$\eqalign{ \phi &= A:X = \frac{A:Y}{\lambda} \cr d\phi &= \frac{\lambda A:dY-A:Y\,d\lambda}{\lambda^2} \cr &= \lambda^{-3}\Big(\lambda^2A - (A:Y)Y\Big):dY \cr \frac{\partial\phi}{\partial Y} &= \lambda^{-3}\Big((Y:Y)A - (A:Y)Y\Big)\cr }$$ Set the gradient to zero and solve $$\eqalign{ (A:Y)Y &= (Y:Y)\,A \cr Y &= \pm\beta A \cr X &= Y/\|Y\|_F = \pm A/\|A\|_F \cr\cr }$$ Admittedly, this is a bit clumsier than the other responses, but all of the good ideas were already taken.