I want to find the operator norm of
$$A = \begin{bmatrix} 2 & 0 \\ 0 & -3 \end{bmatrix}$$
My teacher defines the operator norm as
$$\|A\| = \max_{\|x\| \leq 1} \|Ax\|_2$$
In the problem description, I am instructed to use
$$\|A\| = \sqrt{\lambda_{\max} (A^T A)}$$
A few things that trouble me:
Is $\|A\| = \sqrt{\lambda_{\max} (A^T A)}$ only valid if the operator norm is induced from Euclidean norm $\| \cdot \|_2$?
Why does $\|A\| = \sqrt{\lambda_{\max} (A^T A)}$ hold?
What if $\|A\|$ was induced say for the $1$-norm, or $\infty$-norm, does $\|A\| = \sqrt{\lambda_{\max} (A^T A)}$ still hold?
It is valid only if the operator norm is induced by the Euclidean norm. If you use $||\cdot||_1$ or $||\cdot||_{\infty}$, you'll get different expressions of the operator norm.
In order to show that this equality holds when the norm on $\mathbb{R}^n$ is the Euclidean norm, you can diagonalize $A^TA$ in an orthonormal basis : there exists an orthogonal matrix $P$ and a diagonal matrix $D$ such that $D=P^TA^TAP=(AP)^T(AP)$, and the diagonal coefficients of $D$ are the eigenvalues of $A^TA$, which are $\geq 0$. I'll denote $\sqrt{D}$ the matrix with the square roots of the eigenvalues of $A^TA$ on its diagonal.
You can first show that $||Ax||_2^2=||\sqrt{D}x||_2^2$. So $$\left(\max_{||x||\leq 1}||Ax||_2\right)^2=\max_{||x||\leq 1}||\sqrt{D}x||_2^2$$
Then, for $||x||\leq 1$, $||\sqrt{D}x||^2_2=\sum_{i=1}^n\lambda_ix_i^2\leq\max\{\lambda_i\}$ This implies that $$\left(\max_{||x||\leq 1}||Ax||_2\right)^2\leq \max\{\lambda_i\}$$
For the other inequality, we have, for every $i$, $\lambda_i=||\sqrt{D}e_i||_2^2$, where $e_i$ is the $i$-th vector of the standard basis of $\mathbb{R}^n$. As $||e_i||_2=1$, this implies that
$$\max\{\lambda_i\}\leq \max_{||x||\leq 1}||\sqrt{D}x||_2^2$$