Using Newton's Method in Backward Euler Method

Question

I'm currently looking at this example problem from my course notes (introductory numerical analysis):

and am a bit confused about how to write the equation for Newton's method. From its general form, I would conjecture that it is simply

$$ w_{i+1} = w_i - \frac{f(t_i, w_i)}{f_y(t_i, w_i)} $$

but that seems incorrect as $f$ is a function of multiple variables. Is this correct or should I be considering changes in the variable $t$ as well?

Answer 1

When you write Newton formula as given in your book, what you basically write is that $$f(x)=f(x_0)+(x-x_0)f'(x_0)=0$$ from which $$x-x_0=-\frac{f(x_0)}{f'(x_0)}$$ To stay simple, let us consider now two equations $f(x,y)=0$ and $g(x,y)=0$. We can now write $$f(x,y)=f(x_0,y_0)+(x-x_0)f'_x(x_0,y_0)+(y-y_0)f'_y(x_0,y_0)=0$$ $$g(x,y)=g(x_0,y_0)+(x-x_0)g'_x(x_0,y_0)+(y-y_0)g'_y(x_0,y_0)=0$$ from which $(x-x_0)$ and $(y-y_0)$ can be calculated solving the two linear equations for the two unknowns. As you see, it is very similar but, in the first case, we use the derivative of the equation while, in the second case, we use something slightly more complex (called the Jacobian of the system) which is build from the derivatives of each equation with respect to ech variable.

I shall not enter here in more details. Let me know if this make things clearer to you.

Answer 2

To get the multi-variable analogue, simply replace $\frac{1}{f'(x)}$ with the inverse Jacobian of the system $J^{-1}$, so \begin{equation} \textbf{x}_{i+1} = \textbf{x}_i - J^{-1}\mathbf{f}(\mathbf{x}_i), \end{equation} where the entries of $J$ \begin{equation} j_{ik} = \frac{\partial f_i}{\partial x_k} \end{equation} are all the first partial derivatives of all functions.

Note there are computationally efficient ways of computing the $J^{-1}\mathbf{f}(\mathbf{x}_i)$ which don't involve calculating the Jacobian, though this is beyond the scope of this question.

Answer 3

Staying in your notation, it should be $$ w_{i+1}^{m+1} = w_{i+1}^{m} - F_y(w_{i+1}^{m})^{-1}F(w_{i+1}^{m}) $$ with $F(y)=y-w_i-hf(t+h,y)$, so that $F_y(y)=I-hf_y(t+h,y)$.

---

Using $w_{i+1}^{0}=w_i$ as a crude predictor, the first approximation is \begin{align} w_{i+1}^{1} &=w_i+h(I-hf_y(t+h,w_i))^{-1}f(t_{i+1},w_i) \\ &=w_i+hf(t_{i+1},w_i)+h^2(I-hf_y(t+h,w_i))^{-1}f_y(t+h,w_i)f(t_{i+1},w_i) \end{align} This is related to the Rosenbrock integrators for stiff systems.

---

As better predictor you might take the forward Euler step $$ w_{i+1}^{0}=w_i+hf(t_i,w_i) $$