We define the Moore–Penrose inverse of a matrix $A$ as follows
$$A^+ := \lim_{x \to 0} \, (A^T A + xI)^{-1} A^T$$
and we say that it finds the lowest norm solution of $\|Ax - y\|$. I'm not sure about 2 things:
Why is $(A^T A + xI)$ always invertible?
How does this definition imply $A^+ A = I$?
Sketch for proof of 1): if $\lambda_1,\ldots,\lambda_n$ are the eigenvalues of $A^\top A$, then $\lambda_1+x, \ldots, \lambda_n + x$ are the eigenvalues of $A^\top A + x I$ (why?). Since the $\lambda_i$ are nonnegative (why?) and $x > 0$, we see that $A^\top A + x I$ has no zero eigenvalues.
The pseudoinverse does not satisfy $A^+ A = I$. Check the definition/properties again.