The spectral norm of the difference of two rank-$1$ matrices is equal to the sine of their angle

Question

Can someone help me understand either geometric intuition or mathematical proof that how for two unit vectors $a$ and $b$, the following holds:

$$\left\lVert aa^T - bb^T \right\rVert_2 = \sin \theta$$

where $\theta$ is the angle between $a$ and $b$.

Answer 1

The spectral norm of a $n \times n$ matrix is defined by

$$||A||^2=\max \text{eigenvalue of} \ A^TA$$

If we take $A:=aa^T-bb^T$, we must look for eigenvectors and eigenvalues of

$$B:=A^TA=(aa^T-bb^T)^T(aa^T-bb^T)=(aa^T-bb^T)(aa^T-bb^T)\tag{1}$$

Let us take the following notations:

$c:=\cos \alpha$ and $s:=\sin \alpha$.

Expanding (1):

$$B:=aa^T+bb^T-c(ab^T+ba^T)$$

• $B$ has $n-2$ eigenvectors associated with eigenvalue $0$: indeed 2 dimensional space generated by vectors $\{a,b\}$ has a $(n-2)$ dimensional orthogonal space. Taking any vector $v$ belonging to this orthogonal space, one has indeed :

$$Bv=a(a^Tv)+b(b^Tv)+... = 0$$

• The two other eigenvectors are $a$ and $b$ ; indeed

$$\begin{cases}Ba&=&(aa^T+bb^T-c(ab^T+ba^T))a&=&a+cb-c(ca+b)\\ Bb&=&(aa^T+bb^T-c(ab^T+ba^T))b&=&ca+b-c(a+cb)\end{cases}$$

$$\begin{cases}Ba&=&(1-c^2)a&=&s^2a\\ Bb&=&(1-c^2)b&=&s^2b\end{cases}\tag{2}$$

$a$ and $b$ are eigenvectors associated with eigenvalues $s^2$, giving the final result.

---

If, instead of the spectral norm, we have the Frobenius norm, defined in this way:

$$||A||_F^2=\sum_i\sum_j A_{i,j}^2;$$

the formula is the same but with a factor $\sqrt{2}$ on the RHS. Said otherwise:

$$\text{If} \ A:=aa^T-bb^T, \ \ ||A||_F^2=2 \sin^2 \theta \tag{3}$$

For example, if $a=\binom{1}{0}$ and $b=\binom{0}{1}$ whose angle is $\theta=\pi/2$.

$$A:=aa^T-bb^T=\binom{1 \ \ \ \ \ \ \ 0}{0 \ \ -1}$$

has squared Frobenius norm $2$ which, indeed, is equal to $2 \sin^2 \pi/2.$

Proof of the general case:

The entries of $A:=aa^T-bb^T$ are:

$$A_{i,j}=a_ia_j-b_ib_j.$$

Therefore:

$$||A||_F^2=\sum_i\sum_j (a_ia_j-b_ib_j)^2$$

$$||A||_F^2=\sum_i\sum_j (a_i^2a_j^2+b_i^2b_j^2-2a_ib_ia_jb_j)$$

$$||A||_F^2=\sum_i a_i^2 \sum_ja_j^2+\sum_i b_i^2 \sum_j b_j^2-2\sum_i a_ib_i\sum_j a_jb_j$$

$$||A||_F^2=1+1-2\cos \theta \cos \theta$$

giving finally (3).

Answer 2

You can reduce to the case $a=(1,0, \ldots)$, and $b = (\cos \theta, \sin \theta, 0, \ldots)$ ( by applying an isometry to the pair $a$, $b$). Then you can reduce to the case $\dim V = 2$. Now you operate with concrete $2\times 2$ matrices. We have

$$a a^t = \left(\begin{matrix} 1&0\\0&0\end{matrix}\right) $$ and $$b b^t = \left(\begin{matrix} \cos^2 \theta &\cos \theta \sin \theta \\\cos \theta \sin \theta& \sin^2 \theta \end{matrix}\right) $$

and therefore

$$a a^t - b b^t =\sin \theta \cdot \left(\begin{matrix} \sin \theta &-\cos \theta \\-\cos \theta &- \sin \theta \end{matrix}\right) $$

From the above we can conclude that $a a^t - b b^t = \sin \theta\, \times$ a reflection.

Now it's interesting to figure out what the reflection actually is.

$\bf{Added:}$

Consider the isosceles triangle $CAO$, with heights $OB$ and $CD$. The projection of the vector $\overrightarrow{OC}$ on the rays $OA$ and $OB$ are the vectors $\overrightarrow{OD}$ and $\overrightarrow{OB}$, with difference $\overrightarrow{DB}= \sin \theta \cdot \overrightarrow{OC}$.

Answer 3

$A=aa^T-bb^T$ is a traceless rank-two matrix. It has only two possibly nonzero eigenvalues $\lambda$ and $-\lambda$. Since it is also symmetric, $\lambda$ is real and $\|A\|_2$ is precisely the spectral radius of $A$. Therefore $$ \begin{aligned} \|A\|_2 &=|\lambda| =\sqrt{\frac{(\lambda)^2+(-\lambda)^2}{2}} =\sqrt{\frac{\operatorname{tr}(A^2)}{2}}\\ &=\sqrt{\frac{\operatorname{tr}(aa^Taa^T-aa^Tbb^T-bb^Taa^T+bb^Tbb^T)}{2}}\\ &=\sqrt{1-(a^Tb)^2}=|\sin\theta|.\\ \end{aligned} $$