norm of inverse matrix

Question

Given that $D\gg 1$ (integer) and $\delta\ll 1$ (small positive real). Suppose $v$ is a fixed unit vector, and $u\sim \mbox{Unif}(\sqrt D\mathbb S^{D-1})$. Can we meaningfully bound the norm of $(uv^T+\delta I)^{-1}$ with very high probability, as in $||(uv^T+\delta I)^{-1}||={\tilde O}(\delta^{-1})$, where the hidden constant is allowed to depend on $D$ but only logarithmically? Weyl's perturbation theorem says $||(uv^T+\delta I)^{-1}||\geq \delta^{-1}$; is there maybe an almost matching upper bound, or any hope of having one?

Answer 1

A possibly helpful perspective: by the Sherman Morrison formula, we see that $(uv^T + \delta I)$ is invertible if and only if $u^Tv \neq -\delta$, and in this case we have $$ (uv^T + \delta I)^{-1} = \delta^{-1} I - \frac{uv^T}{\delta^2 + \delta u^Tv}. $$ If we take $\|\cdot\|$ to be the spectral norm, then the triangle inequality yields $$ \|(uv^T + \delta I)^{-1}\| \leq \delta^{-1} + \frac{\sqrt{D}}{|\delta^2 + \delta u^Tv|}. $$ However, as $u^Tv \to -\delta$, the norm of the inverse becomes arbitrarily high.