I want to understand how newton's method is derived from Taylor expansion, and as many answers show that $$f(x+h)=f(x)+h f'(x)+\frac{1}{2} h^2 f''(x)+O\left(h^3\right)$$
and would simply it to : $$f(x+h)=f(x)+h f'(x)+\frac{1}{2} h^2 f''(x)) = 0$$
My question is: Why are we assuming $$f(x+h)=f(x)+h f'(x)+\frac{1}{2} h^2 f''(x)) = 0\quad ?$$
Why not use 1, or 2, or 3 or any number?
Based on my understanding, if we want to calculate the h that will minimize $$ y =h f'(x)+\frac{1}{2} h^2 f''(x)$$, we would need to take a derivative on it for y', instead of assuming $$f(x+h)=f(x)+h f'(x)+\frac{1}{2} h^2 f''(x)) = 0$$？
Thanks!
I'm assuming you are studying Newton's method to find the root of $f(x)$.
https://en.wikipedia.org/wiki/Newton%27s_method
Newton's method to find the root of $f(x)$ is defined by the following iterative process
$$x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} $$
To understand the rationale behind the method, let us assume that the root equals $x_*=x_0+h_*$. Then, by definition, $f(x_*)=f(x_0+h_*)=0$. This is the definition of root.
Newton's method is based on two assumptions:
$f(x_*)=f(x_0+h_*)=0$
$f(x_*) \approx f(x_0) + h_* f'(x_0) + h_*^2 f''(x_0+\theta h_*)/2$
Note that the term $h_*^2$ is assumed to be small, so the contribution of $h_*^2 f''(x_0+\theta h_*)/2$ is small. For instance, if $ h_*=0.01$, then $h_*^2=0.0001$ which is much smaller than $h_*$, so $h_*^2$ multiplied by something can be eliminated. This does not necessarily mean that the second derivative itself is small.
A more rigorous justification as to why the term $h_*^2 f''(x_0 + \theta h_*)/2$ is neglected relates to the fact that by doing so the error of the method decays quadratically with respect to the number of steps
Newton method: quadratic convergence
Then,
$$f(x_*) =0 \approx f(x_0) + h_* f'(x_0) $$
which produces
\begin{equation} h_* \approx -\frac{f(x_0)}{f'(x_0)} \end{equation}
If we denote the right hand side by $h$, we get Newton method, wherein \begin{equation} x_{n+1} = x_n + h = x_n -\frac{f(x_0)}{f'(x_0)} \end{equation}
Please, check the full answer at
Newton iteration converges monotonically