how to use the maximum likelihood function to acquire the maximum likelihood (ML) estimate

Question

I'm reading this book Tracking and Data Association. The author gives an example for estimating an unknown non-random variable $x$ given some observations $z_{j}$ corrupted by Gaussian noise $ w \sim \mathcal{N}(0, \sigma^{2})$ as follows

$$ z = x + w $$

The maximum likelihood function then is $$ \Lambda(x) = p(z|x) = \mathcal{N}(z; x, \sigma^{2}) = \frac{1}{\sqrt{2\pi} \sigma} exp \left(\frac{(z-x)^{2}}{2\sigma^{2}} \right) $$

Now the goal is to acquire $\hat{x}^{ML}$ which is

$$ \hat{x}^{ML}(j) \triangleq \arg\max_x \ p(z_{j} | x ) $$

In Matlab, the following code is to generate $z_{j}$

clear all
clc

x = 1;
sigma = 0.5;
for i = 1:5
    z(i) = x + sigma*randn();
end

The observations are

1.7192    1.1626    0.6225    1.6851    0.1442

The problem is now how I can calculate $\Lambda(x)$ since $x$ is unknown? In the book, the example is for one observation and it is evident that $\hat{x}^{ML} = z_{1}$. Should I start from this value to calculate the rest of $\Lambda(x)$?

Answer 1

This can be done by hand if you know these basic facts:

• the $\arg\max_x \Lambda(x)$ is located at the $x$ that solves $\frac{d}{dx}\Lambda(x)=0$
• the $\arg\max_x \Lambda(x)$ is unaffected by the $log$ function; $argmax_x \Lambda(x) = argmax_x log( \Lambda(x) )$
• if $z$ is a collection of independent samples $z_1, z_2, ... z_n$, then their joint probability is the product of the marginal distributions: $p(z|x)=p(z_1, z_2, ... z_n|x) = \prod_j p(z_j|x)$
• the $log$ of a product is the sum of $log$'s: $log(\prod_j p(z_j|x))=\sum_j log(p(z_j|x))$

Now we can derive an equation for $\hat{x}^{ML}$.

$$log( p(z|x) ) = log(\prod_j p(z_j|x) ) = \sum_j log(p(z_j|x)) $$

$$= \sum_j log( \frac{1}{\sqrt{2\pi} \sigma}) + \sum_j log( exp \left(\frac{(z_j-x)^{2}}{2\sigma^{2}} \right) )$$

$$= \sum_j log( \frac{1}{\sqrt{2\pi} \sigma}) + \sum_j \left(\frac{(z_j-x)^{2}}{2\sigma^{2}} \right) $$

$$\arg\max_x p(z|x) = \arg\max_x log( p(z|x) ) = \arg\max_x \left[ \sum_j log( \frac{1}{\sqrt{2\pi} \sigma}) + \sum_j \left(\frac{(z_j-x)^{2}}{2\sigma^{2}} \right) \right]$$

$$\arg\max_x p(z|x) = \arg\max_x \left[ \sum_j \left(\frac{(z_j-x)^{2}}{2\sigma^{2}} \right) \right]$$

$$0 = \frac{d}{dx} \left[ \sum_j \left(\frac{(z_j-x)^{2}}{2\sigma^{2}} \right) \right]$$

$$0 = \sum_j \frac{-2(z_j-x)}{2\sigma^{2}} $$

$$0 = \sum_j^n -z_j + \sum_j^n x $$

$$\hat{x}^{ML} = \frac{1}{n}\sum_j^n z_j $$

So the maximum likelihood estimator is just the average of the measurements in this case.