Why are these equivalent statements about the largest singular value for matrix $A$? $\sigma \le \,k$ IFF $A^T A \le k^2 I$

Question

I frequently see this statement about any matrix, $A$:

$\|A\|_2 \le k \,\,$ IFF $A^TA \le k^2 I$

where

• $\|A\|_2$ is the L2 norm (or the "spectral norm", or the largest singular value) of A. not to be confused with the euclidean norm of a matrix, because that is different.

• $k \in \mathbb{R_+}$

• $\le$ is a matrix inequality that implies an inequality over the cone $S^n_+$

I do not undastand it.

The following is what I know:

• The matrix $A^TA$ is PSD, which I think should help me in some way (e.g. the eigenvalues are all positive).
• $\lambda_i = \sigma_i^2$ for all nonzero singular values, where $\lambda_i$ is an eigenvalue of $A$

Answer 1

Here's the "only if" part:

For $A\in\mathbb{R}^{m\times n}$, we have that $\|A\|_{2}\le k$, which means that for any $x\in\mathbb{R}^{n}$,

\begin{align} \|Ax\|_{2}^{2}&\le k^{2}\|x\|_{2}^{2}\\ \implies x^{\top}A^{\top}Ax&\le k^{2}x^{\top}x\\ \implies x^{\top}(A^{\top}A-k^{2}I)x&\le 0. \end{align}

Since $x$ was arbitrary, this means that $A^{\top}A-k^{2}I\preceq 0$, or, $A^{\top}A\preceq k^{2}I$.

The other direction is similar.