A problem in method of moment - in my Quiz

Question

Let $X_1,...,X_n$ be an i.i.d. sample from the uniform distribution on ($-\theta$, $\theta$).

(a) Find a method of moments estimator of $\theta$.
By integration of second moment, $\mu_2=E(X^2)=\mu^2+\sigma^2=\frac{\theta^2}{3}$, which implies that $\theta=\sqrt{3\sigma^2}$.

(b) What is the approximate variance of your estimator?
For this question, I have no idea. But if it is a estimator of MLE, we can use asymptotic variance, right?

(c) Denote your estimator by $\hat{\theta}$. Find $E(\hat{\theta})$.

(d) Does $\hat{\theta}$ converge to $\theta$ in probability as $n$ goes to infinity?

Answer 1

First correct your method of moments estimator: what you wrote is wrong.

a) You have simply equated some parameters without any reason. What you should be actually doing is $E(X^2)=\dfrac{\sum_{i=1}^nX_i^2}{n}$. So $\dfrac{\theta^2}{3}=\sum_{i=1}^n\dfrac{X_i^2}{n}$ and so $\hat{\theta}=\sqrt{\dfrac{3\sum_{i=1}^nX_i^2}{n}}$.

b) Yes you are right in saying that you have to find the asymptotic variance. Note that $\dfrac{\sum_{i=1}^nX_i^2}{n}$ is a mean of i.i.d. r.v.'s which finite moments. Let $Y_n=\dfrac{\sum_{i=1}^nX_i^2}{n}$ and $E(X_i^2), Var(X_i^2)$ you can find out. So $\dfrac{\sqrt{n}(Y_n-E(X_i^2))}{\sqrt{Var(X_i^2)}}\to N(0,1)$

Now the function $g(x)=\sqrt{3x}$ is differentiable so $\sqrt{n}(g(Y_n)-g(E(X_i^2))\to N(0,(g'(E(X_i^2)))^2Var(X_i^2))$. Use this to find your Asymptotic variance (look up Delta Method).

c) Find out the distribution of $X_i^2$, then $\sum_{i=1}^nX_i^2$ and then $\sqrt{\sum_{i=1}^nX_i^2}$. These are all you need to find $E(\hat{\theta})$.

d) All the machinery is provided. You should be able to find this out. If you cannot, please comment, stating where you are getting stuck.

Answer 2

Extended comment: MME for $\theta$ in $UNIF(-\theta, \theta)$

This is an interesting problem using the method of moments for estimation (MME). A few comments on MME seem appropriate, if you do not already aware of them:

(1) Biasedness of MME. Many MMEs are unbiased, but not all. (Unbiasedness means that the expected value of the estimator is the parameter to be estimated.) However, when an MME results from a derivation involving nonlinear arithmetic, it is not necessarily unbiased. Here you have taken a square root, So $E(\hat \theta) \ne \theta.$ In a simulation of a million runs with $n = 10$ and $\theta = 5,$ I got $E(\hat \theta) \approx 4.95 < 5 = \theta.$ The bias is rather small for $n = 10,$ and it decreases as $n$ increases.

(2) Better estimator. MMEs are not always the 'best' estimators. There is a better estimat0r here: call it $\tilde \theta.$ It is found by taking the larger of the distance of the minimum of the $X_i$ from zero and the distance of the maximum of the $X_i$ from zero, then multiplying that by $(n+1)/n.$ In a commonly used notation, $\tilde \theta = \frac{n+1}{n}\max(-X_{(1)},X_{(n)}),$ where $X_{(1)}, X_{(2)}, \dots, X_{(n)}$ are the sorted data. (The factor $(n+1)/n$ decreases the bias.) I suspect you will learn about methods of estimation in addition to MME later in your studies, if not already.

(3) Mean square error. Roughly speaking, $\tilde \theta$ tends to be closer to $\theta$ than does $\hat \theta.$ One way to measure closeness is to look at the expected mean square. $EMS(\hat \theta) = E[(\hat \theta - \theta)^2]$ and similarly for $\tilde \theta.$ (For an unbiased estimator, EMS is equal to the variance.)

Sometimes it is convenient to return to the original scale or units of the data by taking square roots to get RMSE. In a million runs with $n = 10$ and $\theta = 5,$ I got $RMSE(\hat \theta) \approx 0.72$ and $RMSE(\tilde \theta) \approx 0.46.$

Addendum: The R code and results for the simulation are shown below. Perhaps most important, you might be able to compare a few analytic results for $\hat \theta$ with the simulated values in the first block of code. You may also want to compare results for $\hat \theta$ and $\tilde \theta.$

 B = 10^6;  n = 10;  th = 5
 DTA = matrix(runif(B*n, -th, th), nr = B)
 ss = rowSums(DTA^2);  th.mme = sqrt(3*ss/n)
 mean(th.mme);  sd(th.mme)
 ## 4.947152   #  Approx expectation of theta hat
 ## 0.7224501  #   its approx SD
 sqrt(mean((th.mme - th)^2))
 #3 0.7243801  #   its approx RMSE

 mn = apply(DTA, 1, min);  mx = apply(DTA, 1, max)
 th.alt = ((n+1)/n)*pmax(abs(mn), abs(mx))
 mean(th.alt); sd(th.alt)
 ## 5.000826   # Approx expectation of theta tilde  
 ## 0.4562502  #   its approx SD
 sqrt(mean((th.alt - th)^2))
 ## 0.4562501  #   its approx RMSE

Note: A third and 'seemingly logical' estimator is $\check \theta = 2\bar Y,$ where $Y_i = |X_i|.$ It is unbiased, but $RMSE(\check \theta) = SD(\check \theta) \approx 0.91,$ considerably worse than either $\hat \theta$ or $\tilde \theta.$