Hessian of log likelihood

153 Views Asked by At

Consider i.i.d random variables $X_1,X_2,\cdots,X_n$ with pdf $f(x;\theta)$, where $\theta\in\mathbb{R}$ is the unknown parameter.

We have

the likelihood function $\mathcal{L}=\displaystyle\prod_{i=1}^{n}f(X_i;\theta)$,

the score function $\mathcal{S}=\frac{\partial\log\mathcal{L}}{\partial\theta}$,

and the Hessian $\mathcal{H}=\frac{\partial^2\log\mathcal{L}}{\partial\theta^2}$.

I wonder whether there exists a pdf $f$ (not to be too trival, for example, $f$ is not dependent on $\theta$) such that $\mathcal{H}=0$.

1

There are 1 best solutions below

0
On BEST ANSWER

Not in the regular case, as if the second derivative is $0$, it means that $f(x;\theta)$ is linear in $\theta$, then the max\sup will be on the boundary of the parametric space, as such the Hessian is irrelevant. Namely, $$ f(x;\theta) = \theta g(x), $$ then $$ f'_{\theta} = g(x), $$ which is independent of $\theta$. Another point of view, is the fact that if it is a "regular" case, then no minimal sufficient statistic exists, i.e., there is no MLE. Thus any monotone function of $\theta$ requires boundary solution, and thus the second derivative has no practical meaning.