Calculating MLE for unknown parameters

78 Views Asked by At

Given a random variable $X$, assume it takes on the values $\{1,2,3\}$ with probabilities $Pr[X=1] = p_1$, $Pr[X=2] = 2p_1$, and $Pr[X=3] = p_2$. $p_1$ and $p_2$ are unknown parameters we must calculate using MLE. We also have observations $D = \{x_1, \cdots, x_n \}$ that are iid from $X$.

First, we must ensure that $Pr[X=i]$ is a valid probability mass function. For it to be valid, $3p_1$ + $p_2$ must add up to $1$. Additionally, $p_1 \ge 0$ and $p_2 \ge 0$.

To calculate $\hat{p}$ using MLE, we must first derive the joint probability $Pr[D; p]$ and the log probability $\log Pr[D; p]$. First, the joint probability $Pr[D; p] = p_1^{n_1} (2p_1)^{n_2} (p_2)^{n_3}$ where $n$ represents the number of observations in $D$. The log probability $\log Pr[D;p] = n_1\log p_1 + n_2\log 2p_1 + n_3\log p_2$.

From here, I am unsure how to calculate $\hat{p_1}$ and $\hat{p_2}$ since there are two parameters in $\log Pr[D;p]$. I tried solving for both $p_1$ and $p_2$, but each variable shows up in the closed form solution of other, which I don't believe is a correct formulation.

Edit: Based on the comments, here is what I have tried (for p_1): $$\nabla \log Pr[D;p] = \frac{n_1 + n_2}{p_1} + \frac{n_3}{1 - 3p_2} = 0$$ $$\Rightarrow (1-3p_1)(n_1+n_2) = -p_1n_3$$ $$\Rightarrow p_1 = \frac{n_1 + n_2}{3n_1 + n_2 - n_3}$$