Likelihood of of a sum of densities

Question

I am trying to find the likelihood, and then the log likelihood for the following function: $f(x)= cf_1(x) + (1-c)f_2(x)$. I am stuck on how to find this likelihood because the two densities are being added. Any tips or hints to get started would be greatly appreciated.

Answer 1

You're working with a mixture model with $c$ and $1 - c$ being the mixing proportions. I'll call them $\pi_1$ and $\pi_2$ instead. Your likelihood function is exactly what you think it would be. Assuming independent samples $x_1, \dots, x_N$, it's $$ L = \prod_{n=1}^N (\pi_1 f_1(x_n) + \pi_2 f_2(x_n)) $$ and the log-likelihood is $$ \log L = \sum_{n=1}^N \log(\pi_1 f_1(x_n) + \pi_2 f_2(x_n)). $$ At this point you'd typically rely on the EM algorithm for maximum likelihood estimation of the parameters in $f_1$, $f_2$, and for $\pi_1$, $\pi_2$. For that you might look for stationary points of $$ \sum_{n=1}^N \log(\pi_1 f_1(x_n) + \pi_2 f_2(x_n)) + \lambda(\pi_1 + \pi_2) $$ where the Lagrange multiplier $\lambda$ is introduced to represent the constraint $\pi_1 + \pi_2 = 1$.