Show that the maximum likelihood estimator aims to maximize the probability of a given event

36 Views Asked by At

I'm trying to show that for a continuous random variable $X$ with value in $\mathbb{R}$, the maximum likelihood estimate maximize the probability of observes $x$. So I know that a maximum likelihood estimate maximize the likelihood function that is $L(x,\theta)=f_{\theta}(x)$. We denote $\hat{\theta}$ this MLE. It follows that

$\forall\theta\in\Theta, f_{\theta}(x)\leq f_{\hat{\theta}}(x)\implies\int_{a}^{b} f_{\theta}(x)dx\leq\int_{a}^{b} f_{\hat{\theta}}(x)dx$ where $b>a$

Thus, we conclude that $\forall\theta\in\Theta, \mathbb{P}_{\theta}(X\in[a,b])\leq\mathbb{P}_{\hat{\theta}}(X\in[a,b])$

I would like to have your advice on my reasoning, to know if something is wrong or can be improved, thank you !

1

There are 1 best solutions below

1
On BEST ANSWER

First, knowing that $f_{\theta}(x) \leq f_{\hat \theta}(x)$ does not imply that $\int_a^b f_{\theta}(x) \, \textrm d x \leq \int_a^b f_{\hat \theta}(x) \, \textrm d x$. The MLE maximizes over different values of $\theta$, but only for the observed $x$, not over different values of $x$. Indeed, for $y \neq x$, it may well be that $f_{\theta}(y) \color{blue}{>} f_{\hat \theta}(y)$.

That said: if $a$ and $b$ are quite close to $x$, then you could make an argument involving the continuity of the likelihood function (w.r.t. $x$) to say that $f_{\theta}(y) \leq f_{\hat \theta}(y)$ for all $y \in [a, b]$. However, that continuity argument is an important part of the argument, and it's missing here. I suppose that would be my suggestion for improvement.

More generally: unless you make some restrictions about $a$ and $b$ (and how close they are to $x$), the result you're trying to prove is false.