MLE of non-numerical variables?

Question

I know how to find the MLE in the usual case where the variable takes numerical values. But what if that is not the case?

Suppose we have a sample of the blood type of $1000$ independent individuals

$$ \begin{array}{c|lcr} \text{Blood Type} & \text{O} & \text{A} & \text{B} & \text{AB}\\ \hline \text{Frequency} & 400 & 250 & 250 & 100 \\ \end{array} $$

And we are given the probabilities $$ \Bbb P(X_i=x_i)= \begin{cases} (1-p)^2 &, & x_i=\text{O} \\ p(1-p) &, & x_i=\text{A} \\ p(1-p) &, & x_i=\text{B} \\ p^2 &, & x_i=\text{AB} \\ \end{cases} $$

Where $X_i$ is the blood type of the $i$-th individual. How do we find the MLE?

In a Bernoulli distribution, we would say that

$$ \Bbb P(X_i=x_i)= \begin{cases} 1-p &, & x_i=0 \\ p & , & x_i=1 \end{cases} =p^{x_i}(1-p)^{1-x_i}$$ Do we use some sort of a similar idea in this case?

Thanks

Answer 1

Let $f(x; p) = \mathbb P(X = x)$ denote the probability mass function and $x_1,\dotsc, x_n\in\{O, A, B, AB\}$ denote the realizations. Then, we obtain the log-likelihood function $$ \begin{align*} L(p) &= \sum_{i=1}^n \log(f(x_i; p)) \\ &= 500 \cdot 2 \cdot \log(1-p) + 2\cdot 250 \cdot (\log(p)+ \log(1-p)) + 100 \cdot 2 \cdot \log(p) \\ &= 1500 \log(1-p) + 700 \log(p). \end{align*} $$

Now, differentiate $L$ to maximize it for $p\in(0,1)$.

Answer 2

The Likelihood can be written as: \begin{align} L(p;\mathbf{X})=\prod_{i=1}^n(1-p)^{2\mathbf{1}_{\{X_i=O\}}}[p(1-p)]^{\mathbf{1}_{\{X_i=A\ \cup\ X_i=B\}}} p^{2(1-\mathbf{1}_{\{X_i=O\}}-\mathbf{1}_{\{X_i=A\ \cup\ X_i=B\}})} \end{align} I know nobody is willing to read that, so define: \begin{align} \alpha_i:=\alpha_i(X_i):=\mathbf{1}_{\{X_i=O\}},\ \ \ \ \ \ \beta_i:=\beta_i(X_i):=\mathbf{1}_{\{X_i=A\ \cup\ X_i=B\}} \end{align} and \begin{align} \alpha:=\sum_{i=1}^n \alpha_i, \ \ \ \ \ \beta:=\sum_{i=1}^n\beta_i \end{align} I hope it is not confusing. So we can write the Likelihood now as: \begin{align} L(p;\textbf{X})=\prod_{i=1}^n(1-p)^{2\alpha_i}[p(1-p)]^{\beta_i}p^{2(1-\alpha_i-\beta_i)} \end{align} We keep writing $L(p;\mathbf{X})$ even though no $X_i$ appears in RHS. That is just to let you know that there is still $X_i$ hidden somewhere in the expression, namely in $\alpha_i$ and $\beta_i$.

One rather works with the log-likelihood $l(p;\textbf{X})$ and that is: \begin{align} l(p;\textbf{X})&=\sum_{i=1}^n 2\alpha_i \log(1-p)+\beta_i\log(p(1-p))+2(1-\alpha_i-\beta_i)\log(p)\\ &=(2\alpha+\beta)\log(1-p)+(2n-2\alpha-\beta)\log(p) \end{align}

Notice that $\lim_{p\to 0^+} l(p;\mathbf{X})=\lim_{p\to 1^-}l(p;\mathbf{X})=-\infty$. Moreover if there is only one stationary point for $p\in (0,1)$ then that must be the global maximum (why?).

We will check the stationary points. So we solve $l'(p;\textbf{X})=0$: \begin{align} (2\alpha+\beta)\frac{1}{1-p}+(2n-2\alpha-\beta)\frac{1}{p}=0 \end{align} Or: \begin{align} p(2\alpha+\beta)+(2-2\alpha-\beta)(1-p)=0 \end{align} Or: \begin{align} p=\frac{2\alpha+\beta-2}{4\alpha+2\beta-2} \end{align} Substitute everything back to finish.