Prove that $A \mapsto A (A + \lambda)^{-1}$ is Frechet differentiable for any positive matrix $A$ and for any $\lambda \gt 0.$

Question

Show that the map $f : A \mapsto A(A + \lambda)^{-1}$ is Frechet differentiable for any positive matrix $A$ and for any $\lambda \gt 0.$ Also find it's Frechet derivative.

Is the space of all positive matrices open subset of the space of all self-adjoint operators? If it is the case then we could have written $f(A + B) - f(A)$ as follows $:$ $$f(A + B) - f(A) = B (A + \lambda)^{-1}- (A + B) (A + B + \lambda)^{-1} B (A + \lambda)^{-1}.$$ From here I would guess that $$Df(A) (B) = B (A + \lambda)^{-1} - A (A + \lambda)^{-1}B (A + \lambda)^{-1}.$$ But for that we need to show that if $\|B\|$ is sufficiently small then $$\|(A + B) (A + B + \lambda)^{-1} B (A + \lambda)^{-1} - A (A + \lambda)^{-1}B (A + \lambda)^{-1}\| = o (\|B\|)$$ which I am unable to do. Could anyone please help me in this regard?

Thanks for investing your valuable time in reading my question.

Answer 1

You are given the function $$f(A)=A(A+\lambda I)^{-1}=I-\lambda(A+\lambda I)^{-1}.$$ To form the Frechet derivative, \begin{align} &f(A+B)-f(A)= \\ &=\lambda(A+\lambda I)^{-1}-\lambda(B+A+\lambda I)^{-1} \\ &=\lambda (A+\lambda I)^{-1}-\lambda(A+\lambda I)^{-1}(B(A+\lambda I)^{-1}+I)^{-1} \\ &=\lambda(A+\lambda I)^{-1}-\lambda(A+\lambda I)^{-1}\sum_{n=0}^{\infty}(B(A+\lambda I)^{-1})^n(-1)^n \\ &\approx \lambda(A+\lambda I)^{-1}(B(A+\lambda I)^{-1}+\cdots) \end{align} So the Frechet derivative is the linear term in $B$ on the right, which is $$ f'(A)B= \lambda(A+\lambda I)^{-1}B(A+\lambda I)^{-1}. $$

Answer 2

Let $x$, $y$ be arbitrary with $x$ invertible. Suppose that $$z = (x^{-1} - y)^{-1}$$ exists. Then

$$z = x + xyx + xyzyx\text{.}$$

If the norm satisfies $\|uvu\|\leq \|u\|^2\|v\|$ for all $u$, $v$, then

$$(1 - \|x\|^2\|y\|^2)\|z\| \leq \|x\|(1+ \|x\|\|y\|)\text{.}$$ If furthermore $\|y\| < \|x\|^{-1}$, then

$$\|z\| \leq \frac{\|x\|}{1-\|x\|\|y\|}$$ so that

$$\|z-x-xyx\| \leq \frac{\|x\|^3\|y\|^2}{1-\|x\|\|y\|}\text{.}$$

Finally, set $x = (A + \lambda)^{-1}$, $y = -B$. Then

$$\|f(A+B)-f(A)-\lambda (A+\lambda)^{-1}B(A+\lambda)^{-1}\| \leq \frac{\lambda\|(A+\lambda)^{-1}\|^3\|B\|^2}{1-\|(A+\lambda)^{-1}\|\|B\|}\text{.}$$