Are functions of valid estimators themselves valid estimators?

Question

My questions : If $\hat{\theta}$ is a valid estimator of $\theta$, does that always mean that $g(\hat{\theta})$ is a valid estimator for $g(\theta)$ ? If it is valid, is it as good or consistent as $\hat{\theta}$? Is there any theorem that links the two estimators (at least for some specific functions $g$)?

Answer 1

It depends on what you mean by valid. 'Unbiasedness'? 'Minimum variance'? 'Minimum mean squared error'?

Estimators based on 'sufficient statistics' tend to have desirable properties. A function of such an estimator is again based on sufficient statistics.

A linear function of a an unbiased estimator is unbiased. For example, if $E(\bar X) = \mu,$ then $E(\bar Y) = \mu _ a,$ where $Y_i = X_i + a,$ and $\mu$ is the mean of the population from which $X_i$ are samples. However, unbiasedness does not survive nonlinear transformations. For example, $E(S^2) = \sigma^2,$ but $E(S) \ne \sigma,$ where $S^2$ is the sample variance and $\sigma^2$ is the population variance.

You can look up some of the terms in 'single quotes' above, if you are not familiar with them.

---

Below are a few demonstrations via simulations in R statistical software. You give no clue about the level of your course or background. I hope you can follow the main thread of the simulations. And maybe you can do analytic proofs of some of the results.

If we have $n=10$ random observations $X_i$ from $Norm(\mu = 100, \sigma = 15).$ Then $E(\bar X) = \mu = 100,\, E(S^2) = \sigma^2 = 225,$ but $E(S) < \sigma = 15.$

We illustrate by averaging $m=10^6$ simulated samples to approximate expectations--accurate to a few significant digits.

m = 10^6;  n = 10;  mu = 100;  sg = 15
x = rnorm(m*n, mu, sg)
DTA = matrix(x, nrow=m)  # sample of size n in each row
a = rowMeans(DTA)        # vector of m sample means
mean(a)
## 99.99559              # aprx 100
v = apply(DTA, 1, var)   # vector of m sample variances
mean(v)
## 225.0379              # aprx 225
s = sqrt(v)              # vector of m sample standard deviations
mean(s)
## 14.59105              # S is biased, mean noticeably below 15.

If we have $n = 5$ observations $Y_i$ from $Unif(0,\, \theta=10),$ Then consider two estimators of $\theta,:$ $T_1 = 2\bar Y$ and $T_2 = \frac{n+1}{n}W,$ where $W = \max(Y_i).$ Then $E(T_1) = \theta = 10.\,$ $E(W) = \frac{n}{n+1}\theta = 0.8333\theta = 8.333$ and $E(T_2) = \theta = 10.$ So $T_1$ and $T_2$ are both unbiased. But $T_2$ can be considered the better estimator because $Var(T_2) < Var(T_1).$

10^6; n = 5; th = 10
y = runif(m*n, 0, th)
DTA = matrix(y, nrow=m)
t1 = 2*rowMeans(DTA)
mean(t1);  var(t1)
## 10.00148   # aprx 10, so T1 is unbiased
## 6.660143
w = apply(DTA, 1, max)
mean(w);  var(w)
## 8.334218   # W seriously biased
## 1.98408
t2 = ((n+1)/n)*w
mean(t2);  var(t2)
## 10.00106   # aprx 10, so T2 is unbiased
## 2.857076   # considerably smaller variance than T1