Why is that the matrix $1$-norm and $\infty$-norm are equal to the operator norm, while the $2$-norm is not?

Question

I had always assumed that the operator norm of a matrix $A$:

$$\|A\|_p = \sup\limits_{\|v\|_p \neq 0}\dfrac{\|Av\|_p}{\|v\|_p}$$

was the same as the norm of a matrix:

$${\displaystyle \|A\|_{p}= \left(\sum _{i=1}^{m}\sum _{j=1}^{n}|a_{ij}|^{p}\right)^{1/p}}$$

However, this post seems to shatter my assumption: 2-norm vs operator norm. Upon further examination, it seems that the operator norm and matrix norm only coincide (=) for the matrix $1$-norm or the matrix $\infty$-norm (and extremely limited cases for matrix $2$-norm).

Why is this so? Is there a theorem that relates the operator norm with matrix $1$ or $\infty$ norm and show their equality?

---

Note:

Answer 1

As mentioned by @user1551 operator norm induced by 1-norm and $\infty$-norm are not equal to matrix 1 and $\infty$ norm. Moreover Forbenius norm or matrix-2 norm is always greater than equal to operator norm induced by 2-norm because operator norm induced by 2-norm is maximum singular value(by definition) and matrix 2-norm is $(\sum_i\sigma_i^2)^\frac{1}{2}$. To see this note :$$\left(\sum _{i=1}^{m}\sum _{j=1}^{n}|a_{ij}|^{2}\right)=trace(A^TA)=\sum_i eigenvalue_i(A^TA)=\sum_i singularvalue_i(A)^2=\sum_i \sigma_i^2$$

Answer 2

No, the vector $1$-norm or $\infty$-norm are not the same as their induced counterparts. E.g. \begin{align*} \|(1,0,0,1)\|_1=2&\ne1=\max_{\|v\|_1\ne0}\frac{\|I_2v\|_1}{\|v\|_1},\\ \|(1,1,1,1)\|_\infty=1&\ne2=\max_{\|v\|_\infty\ne0}\frac{\left\|\pmatrix{1&1\\ 1&1}v\right\|_\infty}{\|v\|_\infty}. \end{align*} In fact, the induced $1$-norm of a matrix $A$ is given by the maximum $1$-norm of all columns. Unless $A$ has $n-1$ zero columns, its induced $1$-norm is always strictly smaller than its vector $1$-norm.

Also, the the induced $\infty$-norm of a matrix $A$ is given by the maximum $1$-norm of all rows. In most cases, it is different from the vector $\infty$-norm of $A$.