Question: Assuming that all matrix inverses involved below exist, show that
$$(\mathbf{A}-\mathbf{B})^{-1}=\mathbf{A}^{-1}+\mathbf{A}^{-1}(\mathbf{B}^{-1}-\mathbf{A}^{-1})^{-1}\mathbf{A}^{-1}$$
in particular
$$(\mathbf{I}+\mathbf{A})^{-1}=\mathbf{I}-(\mathbf{A}^{-1}+I)^{-1}$$
and
$$\det[(\mathbf{I}+\mathbf{A})^{-1}+(\mathbf{A}^{-1}+\mathbf{I})^{-1}]=1$$
On
We have \begin{align} \\&(A^{-1}+A^{-1}(B^{-1}-A^{-1})^{-1}A^{-1})(A-B)\\&=I+A^{-1}(B^{-1}-A^{-1})^{-1}-A^{-1}B-A^{-1}(B^{-1}-A^{-1})^{-1}A^{-1}B\\&=I+A^{-1}[(B^{-1}-A^{-1})^{-1}B^{-1}-I-(B^{-1}-A^{-1})^{-1}A^{-1}]B\\&=I+A^{-1}[(B^{-1}-A^{-1})^{-1}(B^{-1}-A^{-1})-I]B=I+0=I\end{align}
The last two results are straightforward.
On
Proving the first equation seems to be mainly a matter of patience in simplifying the mess of inversions. Here's one approach: Transpose the $A^{-1}$ term to the left and then multiply both sides on the left and right by $A$ to get $$ A[(A-B)^{-1}-A^{-1}]A=(B^{-1}-A^{-1})^{-1}. $$ Then multiply both sides on the right by $B^{-1}-A^{-1}$ to get just $I$ on the right side and, on the left, a product that works out to $$ A(A-B)^{-1}AB^{-1}-AB^{-1}-A(A-B)^{-1}+I. $$ Cancel the final $I$ here against the right side, and transpose the two negative terms to the right, reducing your equation to $$ (A-B)^{-1}AB^{-1}=B^{-1}+(A-B)^{-1}. $$ Abbreviating $A-B$ as $C$ (which I should perhaps have done sooner), and thus writing $A$ as $B+C$, we get the preceding equation into the form $$ C^{-1}(B+C)B^{-1}=B^{-1}+C^{-1}, $$ which is easy to verify by multiplying out the left side. Now just read all the steps in reverse to get a proof of the original equation.
I would expect that this argument can be simplified. Think of it as a stream-of-consciousness record of how I attacked the problem rather than as any sort of polished solution.
On
Checking the identity is simple arithmetic. What's not so simple is discovering it. Surprisingly, this is very easy using formal power series. For $\rm\ a = A':=A^{-1},\,\ b = B',\ $ your identity is
$\rm(1)\quad\ \ a'-b'\ \ \, =\, \ \ a + a(b-a)'a,\ $ so right-multiplying by $\rm\ a'\ $ yields
$\rm(2)\quad{(a'-b')'a'} = 1 + a\color{#90f}{(b-a)'}.\ $ Moving terms in and out of the inverses yields
$\begin{align}\rm(3)\quad (a'-b')'a'&\rm = (a(a'-b'))'\, =\, (1-ab')'\\[.2em] \rm\, \color{#90f}{(b-a)'} &\rm = (b(1-b'a))' = \color{#90f}{(1-b'a)'b'} \end{align}\,\ $ so the above becomes
$\rm(4)\quad (1-ab')' = 1+ a\color{#90f}{(1-b'a)b'}\ $ with proof below, via geometric formal power series
$\begin{align}\rm \rm(5)\quad\! (1-ab')' &\rm= (1-ab')^{-1}\\ &=\ \:\!\rm 1+ \color{#c00}a\color{#0a0}{b'}\! + \color{#c00}a{b'a}\color{#0a0}{b'}\! +\! \color{#c00}a{b'ab'a}\color{#0a0}{b'} +\,\cdots\\ &=\ \:\!\rm 1+ \color{#c00}a (1\, +\, \color{c00}{b'a}\ \ +\ \ \color{0a0}{b'ab'a} +\,\cdots)\color{#0a0}{b'}\\ &=\ \:\!\rm 1+ \color{#c00}a (1\,-\,b'a)'\color{#0a0}{b'}\quad {\bf\small QED}\end{align}$
One can check that this formula works, i.e. if $\rm\:(1-b'a)'$ exists then $\rm\:(1-ab')'\:$ exists and has the above value. But the "proof" is far from rigorous. As Halmos wrote in [1]
Why does it all this work? What goes on here? Why does it seem that the formula for the sum of an infinite geometric series is true even for an abstract ring in which convergence is meaningless? What general truth does the formula embody? I don't know the answer, but I note that the formula is applicable in other situations where it ought not to be, and I wonder whether it deserves to be called one of the (computational) elements of mathematics.
It turns out that there is a way to explain why this works. Due to work by Schutzenberger, Krob and others, one can show that the rational identites that hold true in a ring are all trivial, in the sense that they are consequence of the formula for a geometric power series. For references see the comments to this MO question.
[1] Halmos, P.R. $ $ Does mathematics have elements?
Math. Intelligencer 3 (1980/81), no. 4, 147-153
Start with $$A-B=A(B^{-1}-A^{-1})B\qquad\Rightarrow\qquad(A-B)^{-1}=B^{-1}(B^{-1}-A^{-1})^{-1}A^{-1}. $$ Writing $B^{-1}=\color{blue}{B^{-1}-A^{-1}}+\color{red}{A^{-1}}$, it follows that $$ (A-B)^{-1}=(\color{blue}{B^{-1}-A^{-1}}+\color{red}{A^{-1}})(B^{-1}-A^{-1})^{-1}A^{-1} $$ $$ =(\color{blue}{B^{-1}-A^{-1}})(B^{-1}-A^{-1})^{-1}A^{-1}+\color{red}{A^{-1}}(B^{-1}-A^{-1})^{-1}A^{-1} $$ $$ =A^{-1}+A^{-1}(B^{-1}-A^{-1})^{-1}A^{-1}. $$ The last two properties follow easily.