Maximum Likelihood Estimator of two independent random variables that share a mean

Question

How do I find the MLE of a pair of normal variables $X = N(\bar{x}, \sigma_{X})$, $Y = N(\bar{x}, \sigma_{Y})$? I've found solutions for doing so when sigma is the common variable, but I haven't been able to follow the same process with the mean.

Answer 1

Assuming that $X_1, X_2, \ldots , X_N$ is a random sample from $N(\mu, \sigma_X)$ and $Y_1, Y_2, \ldots , Y_M$ is a random sample from $N(\mu, \sigma_Y)$, then the likelihood is given by,

$f(X_{1:N},Y_{1:M}|\mu, \sigma_X, \sigma_Y) = \prod\limits_{i=1}^{N}f_X(X_i|\mu,\sigma_X)\prod\limits_{j=1}^{M}f_Y(Y_j|\mu,\sigma_Y)$

Now expand $f_X$ and $f_Y$ with gaussian pdfs, take $log$ on both sides and differentiate with respect to $\mu, \sigma_X$ and $ \sigma_Y$, equating the result of each to zero, to get $3$ equations in $3$ variables and solve those equations to get MLEs of $\mu, \sigma_X$ and $\sigma_Y$.

P.S. Let me know if you want me to do above computation. Note that if $\sigma_X$ and $\sigma_Y$ are known then calculations become much easier.

Edit: Here are the equations,

$f(X_{1:n}, Y_{1:n}|\mu, \sigma_X, \sigma_Y) = \prod\limits_{i=1}^{N}\frac{1}{\sqrt{2\pi}\sigma_X}\exp(-\frac{(X_{i}-\mu)^2}{2\sigma_X^2}) \prod\limits_{j=1}^{M}\frac{1}{\sqrt{2\pi}\sigma_Y}\exp(-\frac{(Y_{j}-\mu)^2}{2\sigma_Y^2})$

$\log(f(X_{1:n}, Y_{1:n}|\mu, \sigma_X, \sigma_Y)) = -\frac{N}{2}\log(2\pi\sigma_X^2) - \sum\limits_{i=1}^{N}\frac{(X_{i}-\mu)^2}{2\sigma_X^2} -\frac{M}{2}\log(2\pi\sigma_Y^2) - \sum\limits_{j=1}^{M}\frac{(Y_{j}-\mu)^2}{2\sigma_Y^2}$

$\frac{\partial \log(f(X_{1:n}, Y_{1:n}|\mu, \sigma_X, \sigma_Y))}{\partial \mu} = 0 \\ \Rightarrow \sum\limits_{i=1}^{N}\frac{(X_{i}-\mu)}{\sigma_X^2} + \sum\limits_{j=1}^{M}\frac{(Y_{j}-\mu)}{\sigma_Y^2} = 0 $

$\frac{\partial \log(f(X_{1:n}, Y_{1:n}|\mu, \sigma_X, \sigma_Y))}{\partial \sigma_X^2} = 0 \\ \Rightarrow \frac{N}{2\sigma_X^2} - \sum\limits_{i=1}^{N}\frac{(X_{i}-\mu)^2}{2\sigma_X^2} = 0 \\ \Rightarrow \sigma_X^2 = \frac{1}{N}\sum\limits_{i=1}^{N}(X_{i}-\mu)^2 $

$\frac{\partial \log(f(X_{1:n}, Y_{1:n}|\mu, \sigma_X, \sigma_Y))}{\partial \sigma_Y^2} = 0 \\ \Rightarrow \frac{M}{2\sigma_Y^2} - \sum\limits_{j=1}^{M}\frac{(Y_{j}-\mu)^2}{2\sigma_Y^2} = 0 \\ \Rightarrow \sigma_Y^2 = \frac{1}{M}\sum\limits_{j=1}^{M}(Y_{j}-\mu)^2 $

Turns out that computing a closed form for MLEs of $\mu, \sigma_X$ and $\sigma_Y$ is difficult but based on above three equations a gradient descent based approach can be used to numerically compute the values of the three parameters given the sample values.