On page 47 of Nocedal & Wright's Numerical Optimization, the authors provide a proof for Theorem 3.7 and there is one idea that I didn't understand. The authors prove the following inequality:
$$\|x_k+p_k-x^*\| \le \|x_k+p_k^N-x^*\| + \|p_k-p_k^N\| = O(\|x_k - x^*\|^2) + o(\|p_k\|)$$
and say that a simple manipulation of the above inequality reveals that
$$\|p_k\|=O(\|x_k-x^*\|).$$
Question: What ideas and manipulation are needed to prove that $\|p_k\|=O(\|x_k-x^*\|)$ from the above inequality?
Thank you for taking the time to read my question. Any advice is helpful.
We have: $\|p_k\| \le \|x_k -x^*\| + \|x_k + p_k -x^* \| $, and so, $\|p_k\| \le \|x_k -x^*\| + O(\|x_k-x^*\|^2) + o(\|p_k\|)$.
Choose $K$ large enough so that $o(\|p_k\|) \le {1 \over 2} \|p_k\|$, then for $k \ge K$ you have ${1 \over 2} \|p_k\| \le \|x_k -x^*\| + O(\|x_k-x^*\|^2)$, from which the result follows.