Operator norm calculation for simple matrix

Question

Suppose

$$ A = \left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right) $$

How do I calculate $\|A\|_{\text{OP}}$?

I know the definition of operator norm, but I am clueless on how to calculate it for real example like this. Can somebody please give me a step-by-step instruction on how to do this?

Answer 1

For a matrix $A$, $||A||_{OP}$ is the square root of the largest eigenvalue of $A^TA$, where $A^T$ is $A$'s transpose.

The transpose of $\left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right)$ is $\left( \begin{array}{cc} 1 & 5 \\ 4 & 6 \end{array}\right)$, and hence:

$$A^TA=\left( \begin{array}{cc} 1 & 5 \\ 4 & 6 \end{array}\right)\left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right)=\left( \begin{array}{cc} 26 & 34 \\ 34 & 52 \end{array}\right)$$

The eigenvalues of this matrix are $\{39 + 5\sqrt{53};\space 39-5\sqrt{53}\}$. Therefore, $$||A||_{OP}=\sqrt{39 + 5\sqrt{53}}$$

Answer 2

You need the square root of the largest eigenvalue of $A^TA $.

Or, if you want to do it by definition, it becomes a Lagrange multiplier problem. In fact, in this $2$-dimensional case, it can be reduced to a one-variable optimization.

Concretely, using a bit of first year calculus at the end, you have that \begin{align} \|A\|^2&=\max\{\|Ax\|^2:\ \|x\|^2=1\} =\max\{(x+4y)^2+(5x+6y)^2:\ x^2+y^2=1\}\\ \ \\ &=\max\{26x^2+52y^2+68xy:\ x^2+y^2=1\}\\ \ \\ &=\max\{52-26x^2+68x\sqrt{1-x^2}:\ 0\leq x\leq1\}\\ \ \\ &=39+5\sqrt{53}. \end{align} So $\|A\|=\sqrt{39+5\sqrt{53}}$.

Answer 3

The $2$-norm of matrix $\mathrm A$ can also be computed numerically, say, by solving the following convex optimization problem in $t > 0$

$$\begin{array}{ll} \text{minimize} & t\\ \text{subject to} & \| \mathrm A \|_2 \leq t\end{array}$$

or, using the Schur complement, by solving the following semidefinite program (SDP)

$$\begin{array}{ll} \text{minimize} & t\\ \text{subject to} & \begin{bmatrix} t \, \mathrm I_2 & \mathrm A\\ \mathrm A^\top & t \,\mathrm I_2\end{bmatrix} \succeq \mathrm O_4\end{array}$$

Using CVXPY (with NumPy),

from cvxpy import *
import numpy as np

A = np.array([[1, 4],
              [5, 6]])

# create 2 x 2 identity matrix
I2 = np.identity(2)

# create optimization variable
t = Variable()

# create constraints
constraint1 = [ norm(A,2) <= t ]
constraint2 = [ bmat([[ t*I2,   A],
                      [  A.T,t*I2]]) >> 0 ]

# create optimization problem
optprob = Problem( Minimize(t), constraint1 )

# solve optimization problem
optprob.solve()
print t.value

Using

• constraint1, the minimum is 8.68334897643.

• constraint2, the minimum is 8.68262817347.

From the other answers, the exact value of the minimum is

$$\sqrt{39 + 5 \sqrt{53}} \approx 8.68334897642624$$

and, thus, using constraint1 produces more accurate results.