Let $E$ be complex Hilbert space and $\mathcal{L}(E)$ be the algebra of all bounded linear operators on $E$.
Definition: Let $T \in \mathcal{L}(E)$. The Moore-Penrose inverse of $T$, denoted by $T^{+}$, is defined as the unique linear extension of $(\bar{T})^{-1}$ in $$D(T^{+}) = \mathcal{R}(T)+\mathcal{R}(T)^{\perp},$$ with $\mathcal{N}(T^{+}) = \mathcal{R}(T)^{\perp}$ and $\bar{T}$ is the isomorphism $$\bar{T}:=T|_{{\mathcal{N}(T)}^{\perp}}: {\mathcal{N}(T)}^{\perp} \longrightarrow \mathcal{R}(T).$$ Moreover, $T^{+}$ is the unique solution of the four ''Moore-Penrose equations'': $$TXT = T,\quad XTX = X,\quad XT = P_{N{(T)^{\bot}}}\,\,\mbox{and}\,\,\quad TX = P_{\overline{\mathcal{R}(T)}}{{|}_{D(T^{+})}}.$$
Here $\mathcal{R}(T)$ and $\mathcal{N}(T)$ denote respectively the range and the nullspace of $T$. Also $P_{F}$ denote the orthogonal projection onto $F$.
It is well known that $T^{+}$ is bounded if and only if $T$ has a closed range.
According to this answer, we have
If $A$ is selfadjoint matrix, then $$ A^{+}= \lim_{t \to 0}(A^2+tI)^{-1} A.\;\;(1).$$ Note that this limit is with respect to the strong topology.
If $A$ is not selfadjoint, then $A^+=A^*(AA^*)^+$ (which is equal to $(A^*A)^+A^*$).
How we prove the formula $(1)$?
In a general situation .i.e. when $T$ is an operator and not a matrix, it is possible to make sense of (1) as a strong limit on the domain of $T^+$?
Yes, (1) is true for a hermitian matrix $A$ but not for any matrix.
Proof. Up to a change of orthonormal basis, we may assume that $A=diag(0_p,\lambda_1,\cdots,\lambda_q)$ where $\lambda_i\not=0$ and $p+q=n$.
Then $A^+=diag(0_p,1/\lambda_1,\cdots,1/\lambda_q)$.
On the other hand, if $t\not= 0$ and $t\not=-\lambda_i^2$, then $(A^2+tI)^{-1}A=diag(0_p,\lambda_1/(\lambda_1^2+t),\cdots,\lambda_q/(\lambda_q^2+t))$ and, since there is only a finite number of non-zero eigenvalues, the limit is clearly $A^+$.
To obtain a counter-example for any $A$, choose a random $A$ with rank $<n$. $\square$
EDIT and CORRECTION.
i) Practically, we obtain an approximation $B$ of $A^+$, giving a small value $t_0$ to $t$.
Proposition. The error $||B-A^+||$ is $\approx t_0/a^3$ where $a=\min\{|λ|;λ\in spectrum(A) \setminus{\{0\}}\}$.
Proof. Indeed $\Delta=1/\lambda-\lambda/(\lambda^2+t)=\dfrac{t}{\lambda_i(\lambda_i^2+t)}\sim t/\lambda^3$ when $t\rightarrow 0^+$.
Note also (cf. below) that $(*)$ $\Delta\sim 1/\lambda_i$ when $\lambda_i\rightarrow 0$ and $t$ is fixed.
ii) Now, in a Hilbert, we can consider a bounded, compact, self-adjoint operator $H$ and use the Hilbert-Schmidt theorem in order to diagonalize it. Unfortunately, there is a sequence of non-zero real eigenvalues $\lambda_i, i = 1, ..., N$, with $N=rank(A)$, such that $||\lambda_i||$ is monotonically non-increasing and, when $N=\infty$, $\lambda_i\rightarrow 0$.
In this last case, according to $(*)$, the considered (strong) limit does not exist, even if $t\rightarrow 0^+$.
That works only if $H$ has a finite number of distinct eigenvalues.
iii) Clearly, when $H$ is a bounded self-adjoint operator, it works even worse.