I'm struggling to understand approximating solutions to non linear equations using a Jacobian matrix. I understand intermediate steps, but I'm unsure how everything comes together.
I want to use Newton's method, but by using the Jacobian matrix with forward differences.
For example, I have these two equations that I want to find solutions for: $$ 3y - 2x^3 + 2x^2 - 8x = 0 $$ $$ y^3 - 5x -3 = 0 $$ I'm calling those $F(X)$.
Here is the Jacobian that I think is correct:
$$F^1(x)= \begin{bmatrix}-6x^2+4x-8 & 3\\-5 & 3y^2 \end{bmatrix} $$
I'm picking a point to start at, $$[x, y] = [1, 1]$$
I'm stuck on getting the next iteration of Newton's method, here's what I'm trying:
$$x+h = x -\frac{f(x)}{f^1(x)}$$
I understand the forward difference approximation as: $$f^1(x) = \frac{f(x+h) - f(x)}{h}$$
I'm getting $h$ by, starting with $x = [1,1]$: $$F^1(x)h = -F(x)$$
I'm pretty confused by forward difference, and exactly what it's used for. I'm unsure how it plays a role in Newton's Method.
If someone could provide some guidance I'd appreciate it, thanks!
Let me start considering two equations for two unknown varibles. We need to solve $$f(x,y)=0 \qquad , \qquad g(x,y)=0$$ So, a round a point $(x_0,y_0)$, we have $$f(x,y)=f(x_0,y_0)+f'_x(x_0,y_0)(x-x_0)+f'_y(x_0,y_0)(y-y_0)=0$$ $$g(x,y)=g(x_0,y_0)+g'_x(x_0,y_0)(x-x_0)+g'_y(x_0,y_0)(y-y_0)=0$$ Solving for $(x-x_0)$ and $(y-y_0)$ does not make any problem as soon as the derivatives are known.
In your case, you just need to eveluate the derivatives using forward differences that is to say $$f'_x(x,y)=\frac {f(x+\Delta x,y)-f(x,y)}{\Delta x}$$ $$f'_y(x,y)=\frac {f(x,y+\Delta y)-f(x,y)}{\Delta y}$$ instead of the analytical formulae.
Applied to your case, using $x_0=y_0=1$ and $\Delta x=\Delta y=\frac 1 {100}$, you would get $f_x'(1,1)=-10.0402$ instead of $-10$ and $g_y'(1,1)=3.0301$ instead of $3$, the other derivatives being exact.