Let matrix $A \in \Bbb R^{n \times n}$ have $k$ diagonal elements, where $k < n$, and rest of the elements are zero. I am trying to find the pseudoinverse of $A + \lambda I$ when $\lambda$ approaches zero.
Then $\frac{1}{a_i + \lambda}$ would be the diagonal elements for $i$ going from 1 to $k$ of the pseudo inverse and $\frac{1}{\lambda}$ would be the rest of the diagonal elements. If I put $\lambda$ equal to zero then the pseudo inverse would a matrix with elements of $A$ matrix inverted, but there would be elements going to infinity. But that does not sound right. What is wrong in this logic?
The problem is that the pseudo inverse is not a continuous function on the space of matrices as exactly you've shown. Consider the 1d matrix $(x)$ for $x\in\mathbb R$. Then the pseudo-inverse map is $$ (x)\mapsto\begin{cases}1/x&\text{ if }x\neq 0,\\0&\text{ otherwise.} \end{cases} $$ This is not a continuous at zero, and so we would not expect it to preserve a limit of an element to zero. The same happens with your example when we restrict to the kernel of $A$.