Finding maximum likelihood estimate under certain conditions

143 Views Asked by At

Let $X_1, \ldots X_n$ be an i.i.d. sample from the p.d.f. $f(x; \theta) = 1 + \theta(1 - 2x)$ for $0 \leq x \leq 1$ and $0$ otherwise where $-1 \leq \theta \leq 1$.

(a) Show $\theta_{mle}$, the maximum likelihood estimate of $\theta$, exists and is unique.

(b) What is $\theta_{mle}$ when $\sum_i \left(\frac{1}{X_i} - 2\right) \leq 0$?

(c) What is $\theta_{mle}$ when $\sum_{i} ((1 - 2X_i)/(1 - X_i)) \geq 0$?


I got the loglikelihood function is $\log(\prod_i (1 + \theta(1 - 2x_i))) = \sum_i \log(1 + \theta(1 - 2x_i))$

For $(a)$, I thought that I need to show the second derivative of the expression above w.r.t. $\theta$ is strictly positive. I differentiated to get $\sum_i \frac{(1 - 2x_i)}{1 + \theta(1 - 2x_i)}$, and I differentiated again to get $\sum_i -\frac{(1 - 2x_i)^2}{(1 + \theta(1 - 2x_i))^2}$ but this is always zero or negative.

I'm also pretty clueless on $(b)$ and $(c)$. I can see where the inequality might come into play since there's a $(1 - 2x_i)$ in the numerator for (c) but I'm entirely lost for (b). I tried equating the derivative of the log likelihood function to zero, but I can't figure out how to solve for $\theta$.

Any help is appreciated.

1

There are 1 best solutions below

1
On BEST ANSWER

(a) Firstly note that the log likelihood function $L(\theta) = \sum_{i = 1} \log(1 + \theta(1 - 2x_i))$ is a continuous function for $\theta \in [0,1]$. From the extreme value theorem, we can conclude that $L(\theta)$ attains a maximum value in the interval. This proves the existence. For uniqueness, note that $L''(\theta) < 0$ (except when all $x_i = 1/2$). This implies that $L'(\theta)$ is a strictly decreasing function throughout the interval and consequently can attain the value $0$ for exactly one value of $\theta$ in the interval. If there is solution for $L'(\theta) = 0$, then it has to be unique. If there is no solution for $L'(\theta) = 0$, then it implies that $L'(\theta)$ never changes its sign throughout the interval the maximum is attained at one of the end points. Thus, the MLE exists and is unique.

(b) Note that $L'(-1) = \frac{1}{2} \sum_{i} \left(\frac{1}{x_i} - 2 \right) \leq 0$. Since $L'(\theta)$ is strictly decreasing, $L'(\theta) < 0$ for all $\theta$ and consequently, $L(-1) > L(\theta)$ for all $\theta \in [-1, 1]$. Thus, $\theta_{\mathrm{MLE}} = -1$.

(c) For this case, note that $L'(1) = \frac{1}{2}\sum_{i} \left(\frac{1 - 2x_i}{1- x_i} \right) \geq 0$. Using arguments similar to the ones used in part (b), we can conclude that $\theta_{\mathrm{MLE}} = 1$.