I am brushing up on some statistics before undertaking a Machine Learning course, and I came across the concept of moments and expectation/bias and am completely lost. In particular I can't make heard or tail of the following question:
Consider the random sample $X_1,....X_n$ that is independently and identically distributed, for some population density $f_x$ with finite mean and a positive, non-infinite variance.
$\theta = E[X^2] = var(X) + (E[X])^2 = \sigma^2 + \mu^2 $
Consider the estimator:
$\hat\theta = \{ \frac {1}{n}\sum_{i=1}^nX_i(X_i+1)\} - \{\frac {1}{n}\sum_{i=1}^nX_i\}^2$
Assume the observed values for the random sample are 3,8,11,12,19.
What is the corresponding observed value of $\hat\theta $?
Calculate the expectation and the bias of this estimator as functions of $\sigma$ and $\mu^2$
I believe question 1 to be simple enough, just a case of subbing the observed values from the sample into the formula:
$\hat\theta = \{ \frac {1}{5} (1 \times 2 \}(2 \times 3 \}(4 \times 5 \}(6 \times 7 \}(11 \times 12 \} - (\frac {1}{5}(1+2+4+6+11))^2$
$\hat\theta = 40.4 - 23.04 = 17.36 $
However I have no idea how I am supposed to calculated the expectation and bias in Question 2. Any insight or assistance would be greatly appreciated!
Thanks!
i would use the mean squared error, where $E[\hat\theta]= Var(\hat\theta) + (E[\hat\theta]-\theta)^2 $ the estimator is said to be unbiased if $E[\hat\theta]= Var(\hat\theta)$ ,so $(E[\hat\theta]-\theta)^2 $ is the bias