Maximum Likelihood (ML) estimation when 1 estimator is dependent on the other.

34 Views Asked by At

Let $\mathcal{L}(\theta_1,\theta_2)$ be the log-likelihood function. If I manage to find an estimator for $\theta_1$, as $\hat{\theta}_1=g(\theta_2,data)$. Then, if I want to find a ML estimator for $\theta_2$ do I do $\frac{\partial}{\partial \theta_2} \mathcal{L}(g(\theta_2),\theta_2)$ or, $\frac{\partial}{\partial \theta_2} \mathcal{L}(\theta_1,\theta_2)$ and only afterwards I substitute $g(\theta_2,data)$?

I think that I should do the first option. If so, why?

1

There are 1 best solutions below

0
On BEST ANSWER

If $\hat \theta_1$ is a maximum likelihood estimator, then your first method is called a profile likelihood, and is used when you have a model with "nuisance" parameters that you don't care to report. The latter is the maximum of the joint likelihood. Both should give you the same value of $\hat \theta_2$. If $\hat \theta_1$ is some other estimator, then your choice depends on how the bias and variance of $g(\theta_2,data)$ compares to the MLE of $\theta_2$. The one with the lowest MSE will generally be preferred, but it depends on what types of errors you care more about.