I tried so hard but I am unable to solve this problem.
Find the inverse of the matrix $I+ab^T$ .
Hint: Try the form $cI+dab^T$ and find $c$ and $d$. What happens if $a^T b = −1$?
This is an exercise from A Primer on Linear models by Monahan ( Appendix A.69)
Use the hint and multiply the suggested form by your matrix and set it equal to the identity matrix. $$ (cI+dab^T)(I+ab^T) = cI +(c+d)ab^T+da\overbrace{b^Ta}^{\text{scaler}}b^T=cI +(c+d+db^Ta)ab^T=I $$ Since $ab^T$ has rank at most one and cannot be a (nonzero) multiple of identity, the only way this sum is equal to identity is that the coefficient $(c+d+db^Ta)$ in front of $ab^T$ should be zero, and therefore the coefficient $c$ infront of $I$ has to be $1$.
$$ \implies c=1 \implies 1+d(1+b^Ta)=0. $$
If $b^Ta\neq -1$ then $d = -1/(1+b^Ta)$, and the inverse matrix is given by $$ (I+ab^T)^{-1}=I-(1+b^Ta)^{-1}ab^T. $$ If $a^Tb= b^Ta=-1$, the matrix $I+ab^T$ is not invertible. Here is why: Let's multiply $a$ by $I+ab^T$ $$ (I+ab^T)a = a+ab^Ta=a-a = 0. $$ That is, $a$ is an eigenvector of $I+ab^T$ with eigenvalue $0$. If one of the eigenvalues is zero, the matrix is not invertible.