Finding minimum point of banana function using Newton's Method

Question

I am using the banana function $F(x_1,x_2)=(1-x_1)^2+100(x_2-x_1^2)^2$ over $x_1,x_1 \in \Re$. I am using $f_1(x_1,x_2)=0, f_2(x_1,x_2)=0$ to find the minimum point of $F$ which it is (1,1).

Now I want to implement Newton's method for finding the minimum of $F$ via $f_1=f_2=0$ starting with initial condition $x_0=(1.2,1.2)^T$.

How can I do and how can I use Newton's method for finding the minimum point of $F$ ?

Answer 1

We are given the Rosenbrock Function, sometimes called the banana function with $a = 1, b = 100$ and asked to find the minimum using Newton's method with a starting point $X_0 = (x, y) = (1.2, 1.2)$ as:

$$f(x,y)=(1-x)^2+100(y-x^2)^2$$

Newton's method can be extended for the minimization of multivariable functions.

The iteration is given by:

$$X_{n+1} = X_n - [J_n]^{-1} \nabla f_n$$

where: $$ \nabla f = \begin{bmatrix} \dfrac{\partial f}{\partial x} \\ \dfrac{\partial f}{\partial y} \end{bmatrix} = \begin{bmatrix} 2 (1 - x) - 400 x (-x^2 + y) \\ 200 (-x^2 + y) \end{bmatrix}$$

$$J = \begin{bmatrix} \dfrac{\partial^2 f}{\partial x^2} & \dfrac{\partial^2 f}{\partial x \partial y} \\ \dfrac{\partial^2 f}{\partial y \partial x} & \dfrac{\partial^2 f}{\partial y^2} \end{bmatrix} = \begin{bmatrix} 2 + 800 x^2 + 400 (x^2 - y) & 400 x \\ -400 x & 200 \end{bmatrix}$$

The iterates are:

• $X_0 = \begin{bmatrix} 1.2 \\ 1.2 \end{bmatrix}$
• $X_1 = \begin{bmatrix} 1.19592 \\ 1.4302 \end{bmatrix}$
• $X_2 = \begin{bmatrix} 1.00065 \\ 0.963172 \end{bmatrix}$
• $X_3 = \begin{bmatrix} 1.00058 \\ 1.00115 \end{bmatrix}$
• $X_4 = \begin{bmatrix} 1. \\ 1. \end{bmatrix}$
• $X_5 = \begin{bmatrix} 1. \\ 1. \end{bmatrix}$

Here is a graphical representation of the Newton iterates: