I'm reading the proof of Theorem 5.21 (asymptotic normality of M-estimators) in van den Vaarts book "Asymptotic Statistics" (see the attached picture).
(The theorem assumes that $\hat{\theta}_n \to \theta_0$ in probability as $n\to \infty$; i.e., the estimator $\hat{\theta}_n$ is asymptotically consistent.)
I do not understand the last paragraph of the proof. How did he go from the $\sqrt{n}$-consistency to the last conclusion? In particular,
- I do not see where the inequality in page 53 coming from?
- how the $o_p$ term of the last equation in page 52 vanished (I guess it is represented now with the $o_p$ on the right hand side of that equation).
It seems to me that the presentation (somehow) implies that $$ O_p(1) + o_p(\sqrt{n} \|\hat{\theta}_n - \theta_0 \|) = O_p(1)\quad ? $$ Does this hold? My understanding is that this should instead be $O_p(\max(1,\sqrt{n} \|\hat{\theta}_n - \theta_0 \| ))$.
However, if it does hold, then it will follow that $$\sqrt{n} \|\hat{\theta}_n - \theta_0 \| = O_p(1)$$ (we know that this is true if the LHS converges in distribution; however, this is something that we are trying to prove!) and therefore, $$ o_p(\sqrt{n} \|\hat{\theta}_n - \theta_0 \| ) = o_p(O_p(1)) = o_p(1).$$
I think the reason is that if $(1 + o_p(1))X_n = O_p (1)$ then $X_n =(1 + o_p(1))^{-1} O_P(1) = O_p(1) O_p(1) = O_p(1)$ then second equality comes from the section of van der Vaart book on the $O_p$ notation.
Does it make sense?