Wald test and rejection region

Question

Let $(X_i)_{i\in \mathbb N}$ be a sequence of independent random variable with density function $f(x)=\theta x^{\theta -1}\mathbb I_{(0,1)}(x)$, with $\theta>0$. I know that the maximum likelihood estimator for the parameter $\theta$ is given by $$\hat \theta_n=-\frac{1}{\frac{1}{n}\sum_{i=1}^n\log X_i}.$$ The exercise ask me to find the rejection critical region of the Wald test for the hypotesis $H_0:=\theta=\theta_0$ v.s. $H_1:\theta<\theta_0$.

It is the first time I am studying statistics and for what I know the Wald test tell me that if $H_0$ is true then the distribution of $\frac{(\hat \theta_n-\theta_0)^2}{var (\hat\theta_n)}$ is distributed according to a chi quadro with $1$ degree of fredoom. My questions are the following:

1. How can I compute (if this is really necessary) the $var(\hat\theta_n)$?

2. I guess that the $var(\hat\theta_n)$ will depends on $\theta$. How I can apply the Wald test if it depends on $\theta$?

3. Can I replace $var(\hat\theta_n)$ with the campionary variance?

Answer 1

Under certain regularity conditions which hold in your case you have that \begin{equation} \sqrt{n}\left(\hat{\theta}_{n}-\theta\right)\stackrel{d}{\to}\mathcal{N}\left(0,I\left(\theta\right)^{-1}\right) \tag{1} \end{equation} where $I(\theta)$ is the Fisher Information Matrix given by $$ I(\theta)=-\mathbb{E}\left[\frac{\partial^{2}\log f(x;\theta)}{\partial^{2}\theta}\right] $$ Thus, the Wald statistic in this particular case will be: $$ W=nI(\theta)\left(\hat{\theta}_{n}-\theta\right)\stackrel{d}{\to}\chi^{2}_{1} $$ Recall that the Wald test is an asymptotic test, so it only holds for large sample size $n$. As you may have already noticed, $I(\theta)$ depends on $\theta$, so what one usually does in this case is to replace $\theta$ by its MLE $\hat{\theta}_{n}$. Since $\hat{\theta}_{n}$ is consistent then everything should be allright (under certaing conditions). This is called "plug in" method.

If you wanted to compute the variance of $\hat{\theta}_{n}$ then you have three options:

1. Compute it using the density $f(x;\theta)$ since it is a function of $\left\{X_{1},X_{2},\ldots,X_{n}\right\}$. You may need to compute an ugly convolution to do this, or else find that the sample mean has a known distribution and then compute $Var\left(\frac{1}{\bar{X}}\right)$.
2. Approximate it using (1) such that $Var\left(\hat{\theta}_{n}\right)\approx \frac{1}{nI(\theta)}$. Recall that this will only hold in large samples. So if $n$ is small you should not do this. Since you are studying stats then there is something meaningful about $\frac{1}{nI(\theta)}$: it is the Cramer Rao lower bound for the variance of an unbiased estimator. That is, an unbiased estimator cannot have variance lower than that bound
3. You can do bootstrapping (I am not sure you would find this useful)

I hope this helps. Please let me know if there is something you did not understand.

Answer 2

You can actually explicitly compute the distribution of $\hat{\theta}_n$. Here's how you can achieve that.

Check for yourself that if $X\sim f$ where $f(x)=\theta x^{\theta-1}$ for $x\in(0,1)$ and $f(x)=0$ elsewhere, then $-\ln(X) \sim $ Exponential$(\theta)$. Therefore, if $X_1,...,X_n$ are iid random variables sharing a common density $f$, then $$-\sum_{i=1}^n\ln(X_i)=-\ln(X_1)-...-\ln(X_n) \sim \Gamma(n,\theta)$$ This implies $$-\frac{1}{n}\sum_{i=1}^n\ln(X_i) \sim \Gamma(n,n\theta)$$ which means $\hat{\theta}_n$ possesses an inverse gamma distribution with shape parameter $\alpha=n$ and scale parameter $\beta=n\theta$. It's known that $E(\hat{\theta}_n)=\frac{n\theta}{n-1}$ and $V(\hat{\theta}_n)=\frac{n^2\theta^2}{(n-1)^2(n-2)}$ and, assuming $H_0$ is true, $$\frac{(\hat{\theta}_n-\theta_0)^2}{V(\hat{\theta}_n)}<\chi_{1-\alpha}^2 \iff \theta_0\Bigg(1-\frac{n}{n-1}\sqrt{\frac{\chi_{1-\alpha}^2}{n-2}}\Bigg)<\hat{\theta}_n<\theta_0\Bigg(1+\frac{n}{n-1}\sqrt{\frac{\chi_{1-\alpha}^2}{n-2}}\Bigg)$$ Here $\chi_{1-\alpha}^2$ is the critical value for a random variable possesing a chi$-$squared distribution with $1$ degree of freedom and $\alpha$ is your level of significance.