Does Newton Method with Armijo rule find solution for quadratic in one step?

553 Views Asked by At

Consider a quadratic $f(x)=\frac{1}{2}x^TAx-c^Tx$, with $A \in \mathbb{R}^{n \times n}$ positive definite and the Newton's method with Armijo search applied to minimize $f$ over $\mathbb{R}^n$.

It is known that the pure Newton's method converges to the solution in one step, but how about Newton with Armijo search? Say you start with stepsize $t=1$, before accepting $x^1 = x^0 + td^0$ ($d^0$ the Newton direction), the algorithm should check whether the descent armijo condition holds, namely if

$$ f(x^1)-f(x^0) \leq \alpha \nabla f(x^0)^Td^0.$$ Can the same result be said in this case as well? How does one guarantee that the inequality holds and that the method will actually accept the point $x^1$?

1

There are 1 best solutions below

0
On

Well, I managed to solve this myself but I figured I'm gonna post the answer here anyway, in case someone else wonders about this stuff.

The truth is that the Armijo condition is satisfied for $\alpha \leq \frac{1}{2}$, as

$$\frac{1}{2}\nabla f(x^0)^Td^0 = \frac{1}{2}(Ax^0-c)^T(x^1-x^0) $$ $$ = \frac{1}{2}\left[(x^1)^TAx^0 - (x^0)^TAx^0\right] - \frac{1}{2}c^T(x^1-x^0)$$ $$ = \frac{1}{2}\left[(x^1)^TAx^1 - (x^0)^TAx^0\right] + \frac{1}{2}\left[(x^1)^TAx^0 - (x^1)^TAx^1\right] - \frac{1}{2}c^T(x^1-x^0)$$ $$ = \frac{1}{2}\left[(x^1)^TAx^1 - (x^0)^TAx^0\right] - \frac{1}{2}(Ax^1)^T(x^1-x^0) - \frac{1}{2}c^T(x^1-x^0)$$ $$ = \frac{1}{2}\left[(x^1)^TAx^1 - (x^0)^TAx^0\right] - c^T(x^1-x^0) - \frac{1}{2}(Ax^1-c)^T(x^1-x^0)$$ $$ = f(x^1)-f(x^0) - \frac{1}{2}\nabla f(x^1)^T(x^1-x^0).$$ Since we know that $\nabla f(x^1)=0$, the condition holds for $\alpha \leq \frac{1}{2}$.