What is the Probability density function of MLE

Question

Given a random sample of size N from a population with probability density function $f(x)$ that depends on a paramter $\hat{A}$. Its MLE is the minimun of the random sample.

Now, I have been asked to give the probability density function(p.d.f) of $\hat{A}$. What is this exactly? I have been reading my notes, textbook and internet, but I can't find anything about the p.d.f of a MLE.

Answer 1

MLE estimate is itself a random variable because your MLE estimate is function of your random samples.Now what your asking probably is following $$P(\hat{A}_n = a | x_1,x_2,\cdots,x_n) = n.[1-F(a)]^{n-1} .f(a)$$

where $f$ and $F$ is pdf and CDF of your random variable $X$. this is because $\hat{A}_n = \min \{x_1,x_2,\cdots,x_n\}$ . Following is the proof which gives pdf of minimum of random samples from any distribution

---

let $X$ be any continuous random variables with pdf $f$ and CDF $F$ and $\{X_i\} \sim_{iid} X$ be iid samples.Let random variable $M = \min \{X_1,X_2,\cdots,X_n \}$ then its pdf is given by $p_{M}(x) = n (1-F(x))^{n-1}.f(x)$

proof:

first we will find the CDF of random variable $M$ and find the pdf by differenting it

$$ \begin{align} F_{M}(x) & = P [M \leq x] \\ & = 1-P[M > x]\\ & = 1-P[\min\{x_1,x_2,\cdots,x_n\} > x]\\ & = 1 -P[x_1 > x , x_2 > x , \cdots , x_n >x] \\ &= 1-\prod_{i=1}^n P[X_1> x]\\ &= 1 - \Big[1-F(x) \Big]^n \end{align}$$

from which $p_{M}(x) = \frac{d}{dx} F_M(x) = n [1-F(x)]^{n-1} . f(x)$