I am working on Gauss Newton direction but I have a hard time understanding:
$\ J^tJ p = - J^tr $
What I searched and understood is as follow:
$\ - J^tr $ is in face $\ - \nabla f $ which is like steepest descent direction
but what why is there a $\ J^tJ $ at the begining next to $\ p $ , what does it mean and how could I compute $\ p $ out of this equation?
I also tried to review my linear algebra books but no luck.
Linearising $f$ around a point viz. $f(x)\approx f(a)+\nabla f^t (x-a)$ allows us to solve $f=0$ at first order, viz. $Jp=-r$ where $p:=x-a,\,r:=f(a),\,J:=\nabla f^t$. But how can we solve this? Well, here's the clever bit. Left-multiplying both sides by $J^t$ obtains $J^tJp=-J^tr$. Since $J^t J$ is square, it's invertible unless its determinant is $0$. This allows us to obtain $p$ as $-J^-r,\,J^-:=(J^tJ)^{-1}J^t $. It's easy to forget we can't expect $J$ to be a square matrix, but notice that if it were $J^-$ would simplify to $J^{-1}$, which is why the notation $J^-$ is used. It's called the pseudoinverse of $J$.