I have a "tall" matrix $W\in\mathbb{R}^{N\times n}$, $N>n$, with its operator norm being upper-bounded by some constant $C>0$, i.e., $\|W\|_{2\rightarrow 2}\leq C$. What I am trying to estimate, is the quantity \begin{equation} \|I-aWW^T\|_{2\rightarrow 2},\qquad (1) \end{equation} where $a$ is simply another constant $a>0$. Is it possible to acquire an upper bound of (1) in terms of, e.g., C (and $a$)? Because the simple use of the triangle inequality seems quite coarse.
I'm sorry a priori if it's some elementary stuff, but I haven't practice linear algebra for quite a long time and didn't find anything relevant online...
Because $W$ has rank at most $n$, the same is true for $WW^T$.
The matrix $aWW^T$ is positive semidefinite, so its eigenvalues are nonnegative and its largest eigenvalue is $\|aWW^T\|=a\,\|W\|^2$.
The matrix $I-aWW^T$ is selfadjoint (real, symmetric), and so its operator norm is the largest eigenvalue in absolute value. Its eigenvalues are of the form $1-\lambda$ with $\lambda$ an eigenvalue for $aWW^T$. The largest eigenvalue in absolute value for $I-aWW^T$ is then $$ \max\{\lambda_\max-1, 1-\lambda_\min\}. $$ In this case, because the rank of $aWW^T$ is at most $n<N$, we know that $\lambda_\min=0$. Thus $$ \|I-aWW^T\|\leq\max\{a\,\|W\|^2-1,1\}, $$ and this inequality is sharp.