Why does the parameter estimator not depend on the parameter?

Question

In the proof for the Cramer Rao Inequality, my book writes:

$$E[\hat{\theta}] = \int{\hat{\theta}(\textbf{x})L(\theta;\textbf{x})d\textbf{x}=\theta}$$

Then differentiating both sides of this equation with respect to $\theta$, and interchanging the order of integration and differentiation, gives

$$\int\hat{\theta}\frac{\delta L}{\delta \theta}d \textbf{x}=1$$

This is because $\hat{\theta}$ does not depend on $\theta$.

But why does $\hat{\theta}$ not depend on $\theta$? Because i think $\hat{\theta}$ depends on the random sample which depends on $\theta$

Answer 1

The estimator is a function of the random variables $x_i$, which means the estimator is itself a random variable. In a frequentist framework the parameter, however, is not a random variable; it is just a fixed but unknown constant.

It is in this sense that $\hat{\theta}$ is not a function of $\theta$ (even though you are right that each $x_i$ “depends on” $\theta$ in the sense that the $x_i$ are drawn from a distribution with parameter $\theta$).

For the sake of concreteness, say $X$ has a normal distribution with parameters $\mu$ and $\sigma$. Say we draw an iid sample of size $n$ to estimate $\mu$. Then our estimator, the sample mean $\bar{x}=1/n\Sigma x_i$, is a function of the $x_i$, which are each identically distributed $N(\mu, \sigma^2)$ distributions, but it is not a function of $\mu$ or $\sigma$.

This can all be made absolutely precise in the formalism of measure theory, where the estimator is a measurable function, but the parameters describe the underlying probability measure itself.

Answer 2

It's because $\partial _x$ denotes "explicit" differentiation. For example, Say you have a function $f=n^2$. Then $\partial _n f = 2n$, even if $n=z^2$. In that case, $\partial _z f$ would still yield $0$. $D_x$ and $\partial_x$ do not mean the same thing.

One thing to notice. Although the derivative on the left hand side might be a total derivative ($D_x$, in this case), by application of the Leibniz Integral Rule, the differentiation becomes partial inside the integral sign.

Answer 3

The estimator $\hat\theta$ is first of all a statistic, which by definition, is a function of the sample observations $X_1,X_2,\ldots,X_n$ independent of the parameter of interest ($\theta$ in this case). How can an estimator of an unknown parameter be dependent on the very parameter you are estimating?