Show MLE $\bar Z_n$ of normal $N(0,1)$ has $P(|\bar{Z_n}| < n^{-1/4}) \to 1$.

Question

Recall Let $Z_1, \ldots, Z_n$ be iid $N(\mu, 1), \mu \in \mathbb{R}$. For this simple model, we know that the maximum likelihood estimator (MLE) of $\mu$ is given by the sample mean $\bar{Z}_n=n^{-1} \sum_{i=1}^n Z_i$ and that $$ n^{1 / 2}\left(\bar{Z}_n-\mu\right) \stackrel{\mathcal{D}(\mu)}{\longrightarrow} N(0,1) . $$

Since $n^{1/2}\bar{Z_n}\to N(0,1)$, we can write $n^{1/2}\bar{Z_n} = O_p(1)$. By definition, we have $\forall \varepsilon, \exists M, N, s.t. P(|n^{1/2}\bar{Z_n}| > M) \le \varepsilon, \forall n\ge N$.

Equivalent, I can write $\forall \varepsilon, \exists M, N, s.t. P(|\bar{Z_n}| < n^{-1/2}M) \ge 1-\varepsilon, \forall n\ge N$.

Thus I can see that the support of $\bar{Z_n}$ will be concentrated in an $O(n^{-1/2})$ neighborhood of the origin.

However, the author (Semiparametric theory by Tsiatis) claims that $$P(|\bar{Z_n}| < n^{-1/4}) \to 1,$$ and frequently using the term "increasing probability".

Here is what I have tried:

Our goal is to show $\lim _{n \rightarrow \infty} P\left(\left|\bar{Z}_n\right|>n^{-\frac{1}{4}}\right)=0$. That is, show $\forall \varepsilon, \exists N$ s.t. $\forall n \geqslant N, \quad P\left(\left|\bar{Z}_n\right|>n^{-\frac{1}{4}}\right)<\varepsilon$.

With a fixed $\varepsilon$, by def of $O_p(1)$, $\exists M, N$ s.t. $P\left(\left|\bar{Z}_n\right|>n^{-\frac{1}{2}} M\right)<\varepsilon \quad \forall n \geqslant N$. To achieve the goal, we need $P\left(\left|\bar{Z}_n\right|>n^{-\frac{1}{4}}\right)<P\left(\left|\bar{Z_n}\right|>n^{-\frac{1}{2}} M\right)$

To achieve that, we need $\left|\bar{z}_n\right|>n^{-\frac{1}{4}} \Rightarrow\left|\bar{z}_n\right|>n^{-\frac{1}{2}} M$

To achieve that, we need $\quad n^{-\frac{1}{4}}>n^{-\frac{1}{2}} M$

To achieve that, we need $n^{-\frac{1}{2}} n^{-\frac{1}{2}}>n^{-1/2} M$

To achieve that, we need $\frac{1}{\sqrt{n}}>M$

So we need the condition $n<\frac{1}{M^2}$.

Therefore, it seems that to let the above statement hold, I need to make sure $n$ is not larger than $\frac{1}{M^2}$, which is not in the author's statement.

One can use Hoeffding inequality to show the claim, but I'm wondering where goes wrong with the above proof.

Answer 1

One has $$P(|\bar Z_n|\geq n^{-1/4}) \leq 2\exp(-\sqrt n/2)$$ The thesis follows.

Answer 2

I agree that using the definition of $O_p(1)$ may not be the best way.

Since $Z_1,...,Z_n$ are i.i.d. $N(0,1)$, so $Z_i$ is subGaussian(1), and $\sum_{i=1}^{n} Z_i$ is also subGaussian with parameter $\sqrt{n}$. Hence

$$P\left(\left|\sum_{i=1}^n Z_i \right| \ge t\right) \le 2 e^{\frac{-t^2}{2n}}.$$

From above, we have $$P\left(\left|\frac{1}{n}\sum_{i=1}^n Z_i \right|\ge n^{-1}t\right)\le 2 e^{\frac{-t^2}{2n}}.$$

If we let $t = (-2\log(\frac{\varepsilon}{4}))^{1/2}\sqrt{n}$, then we have $$P\left(\left|\frac{1}{n}\sum_{i=1}^n Z_i \right|\ge n^{-1/2}\left(-2\log\left(\frac{\varepsilon}{4}\right)\right)^{1/2} \right)\le \frac{\varepsilon}{2}<\varepsilon$$

In the textbook, it says that $P(|\frac{1}{n}\sum_{i=1}^n Z_i |\ge n^{-1/4})\to 0$. That is, $\forall \varepsilon >0, \exists N s.t. \forall n\ge N, P(|\frac{1}{n}\sum_{i=1}^n Z_i |\ge n^{-1/4})\le \varepsilon.$

Notice that the event $|\frac{1}{n}\sum_{i=1}^n Z_i |\ge n^{-1/4}$ implies the event $|\frac{1}{n}\sum_{i=1}^n Z_i |\ge n^{-1/2}$, therefore,

$$P\left(|\frac{1}{n}\sum_{i=1}^n Z_i |\ge n^{-1/4}\right) \le P\left(|\frac{1}{n}\sum_{i=1}^n Z_i |\ge c n^{-1/2}\right)\le\varepsilon$$ for some $c$.