What are the definitions of the Matrix norms?

Question

Pardon if this question is a bit uninspiring, but the few resources I've found seem to have different definitions. Can someone please provide a definition of the $L_1$, $L_2$, $L_{\infty}$, and $L_p$ norms for Matrices? If you are writing a formal definition would you please also explain it in laymans terms as well; thanks.

Answer 1

Let $p \geq 1$. The $p$-norm of a real $m \times n$ matrix $A$ is the matrix norm induced by the vector $p$-norm. In other words: $$ \|A\|_p = \sup \{ \|Ax \|_p \mid \| x \|_p =1\}. $$ It tells you how much the $p$-norm of a vector can be amplified when you multiply by $A$.

You can show that $\| A \|_1$ is the maximum absolute column sum of $A$, $\| A \|_\infty$ is the maximum absolute row sum of $A$, and $\| A \|_2$ is the largest singular value of $A$.

---

Further comments: Here is a brief explanation or intuition for the above facts. To see that $\| A \|_1$ is the maximum absolute column sum, we can think about how to select $x$ so that $\| Ax \|_1$ is as large as possible. In selecting the components of $x$, we have a "budget" constraint that $\| x \|_1 = 1$. $Ax$ is a linear combination of the columns of $A$, and it makes sense to spend the entire budget on the best column of $A$.

To make $\| Ax \|_\infty$ as large as possible, subject to the constraint that $\|x\|_\infty = 1$, note that $Ax$ has one component for each row of $A$. The best choice of $x$ will have all components equal to $\pm 1$.

If $A = U \Sigma V^T$ is the SVD of $A$, then $\| Ax \|_2 = \| U \Sigma V^T x \|_2 = \| \Sigma y \|_2$, where $y = V^T x$. To make $\|Ax\|_2$ as large as possible, subject to $\|x\|_2 = \|y\|_2 = 1$, we should arrange for the entire mass of $y$ to be concentrated in one component, corresponding to the largest diagonal element of $\Sigma$.