Interpretation of the likelihood function

109 Views Asked by At

Let $\mathbf{X} = (X_1, \ldots, X_n)$ be a sample of i.i.d. random variables ($X_i \sim P_\theta$) and $\mathbf{x}=(x_1,\ldots,x_n) \in \mathbb{R}^n$ – its realization.
I understand the following definition of the likelihood function:

$$L(\mathbf{x}; \theta) = \prod_{i=1}^n f(x_i; \theta).$$

Here likelihood function is just a value of a joint probability density (joint pmf in discrete case) of a random vector $\mathbf{X}$ (where $X_i \sim P_\theta$) calculated at point $\mathbf{x}$.

But in some statistics textbooks the definition of the likelihood function is given as follows:

$$L(\mathbf{X}; \theta) = \prod_{i=1}^n f(X_i; \theta)$$

i.e. in this case likelihood function is a random variable because $X_i = X_i(\omega)$ are random variables.

Ok, I can interpret $L(\mathbf{X}; \theta)$ as a random variable in the case of an absolutely continuous random vector $\mathbf{X}$.

But what is the correct interpretation of $L(\mathbf{X}; \theta)$ in the case of discrete $\mathbf{X}$?
I think in this case we have ($p$ is a pmf) $$L(\mathbf{X}; \theta) = \prod_{i=1}^n p(X_i; \theta) = \prod_{i=1}^n P_\theta(X_i = X_i) = 1.$$

But this expression is meaningless because it is always equal to one...