Doubt in applying Infeasible start Newton method to a really simple example.

69 Views Asked by At

I am looking to apply one iteration of the infeasible start Newton method to the example below:

$$ \min f(x,y,z )= x^4/4 + y^2 + z^2 \\ \text{subject to } \,\,\,\,\,\,\,\,\,\,\,\ x + y + z = 2.$$

I have made a prior post about this same method, because I was having some trouble understand it. You can find this post here.

My attempt of solution. Since nothing is said about initial approximations, I took the free will to take $x^0 = (0,0,1)$ and $v^0=0$ (from my understand, $v^0$ must be an element of $\mathbb R$ since $A \in \mathbb R^{1 \times 3}$ - you can check this reasoning in the answer to my prior post).

Firstly, we must calculate $\Delta x^0$ and $\Delta v^0$ from the system

$$ \begin{bmatrix} \nabla^2f(x^0) & A^\top \\ A & 0 \end{bmatrix} \begin{bmatrix} \Delta x^0 \\ v^0 + \Delta v^0 \end{bmatrix} = \begin{bmatrix} \nabla f(x^0) \\ Ax^0 - b \end{bmatrix}.$$ It is of trivial observation that, since $v^0 = 0, \nabla f(x^0) = (0,0,2), Ax^0 - b = -1$ and $\nabla^2f(x^0) = \begin{bmatrix} 0 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{bmatrix}$ the system above yields

$$ \begin{cases} (0,2\Delta x_2^0,2\Delta x_3^0) + (\Delta v^0, \Delta v^0, \Delta v^0) = (0,0,2) \\[.3cm] \Delta x_1^0 + \Delta x_2^0 + \Delta x_3^0 = -1 \end{cases} \Leftrightarrow \begin{cases} \Delta v^0 = 0 \\[.2cm] \Delta x_2^0 = 0 \\[.2cm] \Delta x_3^0 = 1 \\[.2cm] \Delta x_1^0 = -2 \end{cases}.$$ Hence, we have concluded that $\Delta v^0 = 0$ and $\Delta x^0 = (\Delta x_1^0, \Delta x_2^0, \Delta x_3^0) = (-2,0,1).$

Now, we must apply backtracking line search to $\| r \|_2:$ Initially, let $t:=1.$ We have that

$$ r(x^0 + t\Delta x^0, v^0 + t \Delta v^0 ) = r((-2,0,2),0) = (\nabla f(-2,0,2) + A^\top\cdot 0, A\cdot(-2,0,2)-b) = ((-8,0,4),-2).$$

My first doubt. Now I am supposed to compute $\| ((-8,0,4),-2 \|.$ Should I just think of this as the norm of the vector $\| (-8,0,4,-2) \| ?$ If no, how am I supposed to calculate such norm?

My second doubt. In this case I had that $v^0$ must be a real number so I assumed that the Newton dual step ($\Delta v^0$) must also be a real number. Is this always the case? I.e., the newton primal and dual step dimensions always agree with the dimensions of $v$ and $x$ ?

After this, we must continue with some calculations but I won't extend myself here since I believe my doubts are explained above.

Thanks for any help in advance.