Three Questions on Interpreting the Outcome of the Probability Density Function

Question

I have three questions: What is the interpretation of the outcome of the Probability Density Function (PDF) at a particular point? How is this result related to probability? What we exactly do when we maximize the likelihood?

To better explain my questions:

(i) Consider a continuous random variable $X$ with a normal distribution such that $\mu=1.5$ and $\sigma^2=2$. If we evaluate the PDF at a particular point, say $3.4$, using the formula:

$$f(X)=\frac{1}{{ \sqrt {2\pi \sigma^2 } }} e^{\frac{- \left( {X - \mu } \right)^2}{{ 2\sigma^2 }}} ~,$$

we get $f(3.4)=0.1144$. How we interpret this value?

(ii) I previously read that the result of 0.1144 is not necessarily the probability that $X$ takes the value of $3.4$. But how the result is related to probability concept?

(iii) Consider a sample of the continuous random variable $X$ of size $N=2.5$, such that $X_{1}=2$ and $X_{2}=3.5$. We can use this sample to maximize the log-likelihood:

$$\max \ln L(\mu,\sigma|X_{1},X_{2}) = \ln f(X_1) + \ln f(X_2) $$

If $f(X)$ is not exactly a probability, what are we maximizing? Some texts detail that "we are maximizing the probability that a model (set of parameters) reproduces the original data". Is this phrase incorrect?

Answer 1

The probability density function is the unsigned derivative of the cumulative probability function.

$$f_{\small X}(x)=\begin{vmatrix}\dfrac{\mathrm d ~~}{\mathrm d x}\mathsf P(X\leq x)\end{vmatrix}$$

It may be considered the "gradient of the tangent" of the curve; that is, the "rate of change" of accumulation of probability, as value for the continuous random variable increases.

So you are maximising the amount the parameters contribute to immediate accumulation of probability around the data points.

Answer 2

I'm not an expert, but this is the way I understand it. Denote the cumulative distribution function (CDF) by $$ F_{\small{X}}(x) = \int_{-\infty}^x f_{\small{X}}(t) \ dt.$$

(i) For small $\varepsilon$, $$f_{\small{X}}(x) \approx \frac{F_{\small{X}}(x+\varepsilon) - F_{\small{X}}(x)}{\varepsilon},$$ so $$F_{\small{X}}(x+\varepsilon) \approx F_{\small{X}}(x) + \varepsilon\cdot f_{\small{X}}(x). \ (*)$$ That is, the PDF $f_{\small{X}}(x)$ gives the "rate of change" of the CDF $F_{\small{X}}(x)$.

To illustrate using your example, we can approximate $F_{\small{X}}(3.401)$ using $(*)$ above. We get \begin{align} F_{\small{X}}(3.401) & \approx F_{\small{X}}(3.4) + 0.001 \cdot f_{\small{X}}(3.4)\\ &= 0.910445404 + 0.001 \cdot 0.114404814 \\ &= 0.910559808. \end{align} Using Excel, we see that $F_{\small{X}}(3.401) = 0.910559754$, which is very close to our approximation. (I also used Excel to compute $F_{\small{X}}(3.4)$.)

(ii) As you said, $f_{\small{X}}(3.4)$ is not the probability that $X=3.4$. In fact, the probability that $X=3.4$ if $X$ is a continuous random variable is $0$ since $$\int_{3.4}^{3.4} f_{\small{X}}(t) \ dt=0.$$ In general, the probability that $X=x$, where $x$ is a real number, is $0$.