I am looking at a slides from a neural network class and there is a brief discussion on the inverse of an over-determined and under-determined system. I understood the left inverse of an over-determined system. However, I am confused on one part of the inverse of the under-determined system.
The system is:
$b=Ax$
Let A be an mxn matrix with full row rank r=m. The matrix $AA^T$ is an $m\times m$ matrix. This is a non-singular symmetric matrix and the inverse exists. Hence,
$(AA^T)^{-1}AA^T = I$
Now, the slides jump to this equation:
$(AA^T)^{-1}A^TAx=(AA^T)^{-1}A^Tb$ // I don't understand how this is possible when the dimensions don't match
and from there it says it follows:
$x=A^T(AA^T)^{-1}b=A^+b$ // I don't understand how it gets to this part as well.
and finally:
$A^{-1}_{right\ inverse}=A^+=A^T(AA^T)^{-1}$
Can you help me clarify the steps?