A convex optimisation problem involving the Euclidean norm

Question

Any ideas on how to approach the following optimisation problem?

$$\begin{array}{ll} \text{maximize} & \|Ax\|_2^2+\|Bx\|_2^2+\|Cx\|_2^2 \\ \text{subject to} & \|x\|_2 = 1\end{array}$$

Answer 1

The objective function is

$$\begin{array}{rl} \|\mathrm A \mathrm x\|_2^2 + \|\mathrm B \mathrm x\|_2^2 + \|\mathrm C \mathrm x\|_2^2 &= \mathrm x^T \mathrm A^T \mathrm A \mathrm x + \mathrm x^T \mathrm B^T \mathrm B \mathrm x + \mathrm x^T \mathrm C^T \mathrm C \mathrm x\\\\ &= \mathrm x^T (\mathrm A^T \mathrm A + \mathrm B^T \mathrm B + \mathrm C^T \mathrm C) \mathrm x\\\\ &\leq \lambda_{\max} (\mathrm A^T \mathrm A + \mathrm B^T \mathrm B + \mathrm C^T \mathrm C) \|\mathrm x\|_2^2\end{array}$$

If $\|\mathrm x\|_2 = 1$, then

$$\|\mathrm A \mathrm x\|_2^2 + \|\mathrm B \mathrm x\|_2^2 + \|\mathrm C \mathrm x\|_2^2 \leq \lambda_{\max} (\mathrm A^T \mathrm A + \mathrm B^T \mathrm B + \mathrm C^T \mathrm C)$$

Answer 2

Since you are multiplying $x$ by all these matrices they have the same number of columns. So it is ok to stack them to a single matrix $D$. Then $\|Dx\|_2^2=\|Ax\|_2^2+\|Bx\|_2^2+\|Cx\|_2^2$. Suppose we have a singular value decomposition (SVD) of $D$ $$USV^t=D.$$ Then if $v$ is the first column of $V$ it has the property that it maximizes your problem under the given constraint $\|v\|_2=1$.

Here is a matlab program that compares this to Rodrigo de Azevedo's answer:

clc

N = 30;
n1 = randperm(N); n1 = n1(1);
n2 = randperm(N); n2 = n2(1);
n3 = randperm(N); n3 = n3(1);
m1 = randperm(N); m1 = m1(1);

A = rand(n1,m1);
B = rand(n2,m1);
C = rand(n3,m1);

D = [A;B;C];

[U,S,V] = svd(D);
v = V(:,1);

[norm(D*v)^2, max(eig(A'*A+B'*B+C'*C))]

Answer 3

You are computing the matrix $2$-norm of the matrix $\begin{bmatrix} A \\ B \\ C \end{bmatrix}$. So just use any appropriate method from numerical linear algebra for computing the norm of a matrix.

In Matlab you could use the normest or norm function.