Norm ratio: find the impact of diagonal pertubations

88 Views Asked by At

Given the matrix

$$P(D)=I-A(A^HA+D)^{-1}A^H$$

that is parameterized on the diagonal matrix $D$ whose diagonal entries are not necessarily the same, and $A$ is full column rank, I want find the impact of the diagonal matrix $D$ on the ratio between the squared Euclidean norms

$$r=\frac{||P(0)\mathbf{a}||^2_2}{||P(D)\mathbf{a}||^2_2}$$

Obviously, when the entries of $D$ are small as compared to the corresponding diagonal entries of $A^HA$,

$$r\to 1$$

But what is the difference between the true value of $r$ and 1. Can the difference be writen in terms of $D$, $\mathbf{a}$ and $A$? Or can the difference be estimated, say, for example, when the trace of $D$ is 10% of the trace of $A^HA$, what is the difference between $r$ and 1?

The only way I can think of is to find the derivative of $P(D)$ with respect to each diagonal entry of $D$ and using the Taylor expansion, but the procedure is tedious and not intuitive.

Can you please help? Thank you in advance.

1

There are 1 best solutions below

0
On

The singular value decomposition (SVD) scheme can help some further simplifications, which, I hope, is helpful enough.

Since $A$ is full column rank, its SVD can be written as $$ A=U\begin{bmatrix}D_1\\0\end{bmatrix}V, $$ where $U_{m\times m}$ and $V_{n\times n}$ are unitary, $D_1$ is an $n\times n$ diagonal and invertible matrix, and $0_{(m-n)\times n}$ is a rectangular zero matrix. Henceforth, we will omit the matrix orders, since they will be clear from the context. Through replacement, we achieve $$ I-A(A^HA+D)^{-1}A^H{ = I-U\begin{bmatrix}D_1\\0\end{bmatrix}V(V^H\begin{bmatrix}D_1&0\end{bmatrix}U^HU\begin{bmatrix}D_1\\0\end{bmatrix}V+D)^{-1}V^H\begin{bmatrix}D_1&0\end{bmatrix}U^H \\= I-U\begin{bmatrix}D_1\\0\end{bmatrix}V(V^HD_1^2V+D)^{-1}V^H\begin{bmatrix}D_1&0\end{bmatrix}U^H \\= I-U\begin{bmatrix}D_1\\0\end{bmatrix}(D_1^2+VDV^H)^{-1}\begin{bmatrix}D_1&0\end{bmatrix}U^H \\= I-U\begin{bmatrix}(I+D_1^{-1}VDV^HD_1^{-1})^{-1}&0\\0&0\end{bmatrix}U^H \\= U\begin{bmatrix}I-(I+D_1^{-1}VDV^HD_1^{-1})^{-1}&0\\0&I\end{bmatrix}U^H . } $$ Therefore, by defining $\mathbf{b}=U^H\mathbf{a}$ and $\mathbf{b}=\begin{bmatrix}b_1\\b_2\end{bmatrix}$ where $b_1$ is $n\times 1$ and $b_2$ is $(m-n)\times 1$, we obtain $$ ||P(D)\mathbf{a}||_2^2{= \left|\left|U\begin{bmatrix}I-(I+D_1^{-1}VDV^HD_1^{-1})^{-1}&0\\0&I\end{bmatrix}U^H\mathbf{a}\right|\right|_2^2 \\= \left|\left|\begin{bmatrix}I-(I+D_1^{-1}VDV^HD_1^{-1})^{-1}&0\\0&I\end{bmatrix}\mathbf{b}\right|\right|_2^2 \\= \left|\left|\begin{bmatrix}I-(I+D_1^{-1}VDV^HD_1^{-1})^{-1}&0\\0&I\end{bmatrix}\begin{bmatrix}b_1\\b_2\end{bmatrix}\right|\right|_2^2 \\= \left|\left|\begin{bmatrix}[I-(I+D_1^{-1}VDV^HD_1^{-1})^{-1}]b_1\\b_2\end{bmatrix}\right|\right|_2^2 \\= \left|\left|[I-(I+D_1^{-1}VDV^HD_1^{-1})^{-1}]b_1\right|\right|_2^2 + \left|\left|b_2\right|\right|_2^2, } $$ where without loss of generality we can assume $||b_2||_2=1$, since $\frac{||P(0)\mathbf{a}||_2^2}{||P(D)\mathbf{a}||_2^2}$ is invariant under scaling $\mathbf{a}$. Finally, $$ \frac{||P(0)\mathbf{a}||_2^2}{||P(D)\mathbf{a}||_2^2} = \frac{1}{\left|\left|[I-(I+D_1^{-1}VDV^HD_1^{-1})^{-1}]b_1\right|\right|_2^2+1} . $$ The latter equation cannot be simplified generally, but under some special cases.

Special Case: $D=\alpha I$

When $D$ is a multiple of the identity matrix, we can proceed further and write $$ \frac{||P(0)\mathbf{a}||_2^2}{||P(D)\mathbf{a}||_2^2} {= \frac{1}{\left|\left|[I-(I+\alpha D_1^{-2})^{-1}]b_1\right|\right|_2^2+1} \\= \frac{1}{1+\alpha^2\sum_{i=1}^n\left[\frac{b_i}{\lambda_i^2+\alpha}\right]^2}, } $$ where $\lambda_i$ is the $i$-th eigenvalue of $A$ and $b_i$ is the $i$-th component of $\mathbf{b}$.

The following figure, illustrates its behavior for $n=10$, $b_i\sim\text{Uniform}(0,1)$ and $\lambda_i\sim\text{Uniform}(0,2)$:

enter image description here