Fisher Information Matrix for Weibull Distribution...

Question

I wish to find the Fisher Information Matrix for the Weibull Distribution... I face two difficulties,

1. I can't find any sufficient guide in internet to lead me to derive the Fisher Information Matrix... if have, could you share it to me...
2. I not able to ensure that everything I have proved below is valid or not... but the below are my workings..

Given that the pdf of Weibull Distribution is $$f(x;k,\lambda)=\frac{k}{\lambda}\left(\frac{x}{\lambda}\right)^{k-1}e^{-\left(\frac{x}{\lambda}\right)^k} $$

Likelihood function :

$$ L(k,\lambda|x_i)=k^n\lambda^{-nk}\,\text{exp}\left[(k-1)\sum_{i=1}^n \text{ln}(x_i)-\sum_{i=1}^n \left(\frac{x_i}{\lambda}\right)^k\right]$$

Log-likelhood function :

$$\frac{\delta L}{\delta k}=\frac{n}{\lambda}-n\text{ln}\lambda+\sum_{i=1}^n \text{ln}\left(x_i\right)-\sum_{i=1}^n \left( \frac{x_i}{\lambda}\right)^k$$

$$\frac{\delta L}{\delta \lambda}=-\frac{nk}{\lambda}+\frac{k}{\lambda^{k+1}}\sum_{i=1}^n \left(x_i\right)^k$$

Second Difference...

$$\frac{\delta^2 L}{\delta k^2}=-\frac{n}{k^2}-\sum_{i=1}^{n}\left(\frac{x_i}{\lambda}\right)^k \left( \text{ln}\left( \frac{x_i}{\lambda} \right)\right)^2$$

$$\frac{\delta^2 L}{\delta \lambda^2}=\frac{nk}{\lambda^2}+ \frac{k(k+1)}{\lambda^{k+2}} \sum_{i=1}^n \left(x_i \right)^k$$

$$ \frac{\delta^2 L}{\delta k \delta \lambda}=\frac{\delta^2 L}{\delta \lambda \delta k}=-\frac{n}{\lambda}+\frac{1}{\lambda} \sum_{i=1}^n \left[ \left(\frac{x_i}{\lambda}\right)^k \left( 1+k\, \text{ln}\left( \frac{x_i}{\lambda} \right) \right) \right] $$

Given that the Fisher Information Matrix have to be in the form of

$$ I(k,\lambda) = -E \begin{bmatrix} \frac{\delta^2 L}{\delta k^2} & \frac{\delta^2 L}{\delta k \delta \lambda}\\ \frac{\delta^2 L}{\delta \lambda \delta k} & \frac{\delta^2 L}{\delta \lambda^2} \end{bmatrix}$$

Next, $$E\left(\frac{\delta^2L}{\delta k^2} \right)\\=E\left(-\frac{n}{k^2}-\sum_{i=1}^{n}\left(\frac{x_i}{\lambda}\right)^k \left( \text{ln}\left( \frac{x_i}{\lambda} \right)\right)^2 \right) \\=E\left(-\frac{n}{k^2}\right) - E\left( \sum_{i=1}^{n}\left(\frac{x_i}{\lambda}\right)^k \left( \text{ln}\left( \frac{x_i}{\lambda} \right)\right)^2\right)\\=-\frac{n}{k^2} - \frac{1}{\lambda^k} \sum_{i=1}^nE\left( (x_i)^k \left( \text{ln}\frac{x_i}{\lambda} \right)^2 \right)\\=...$$

And I am stuck for $E\left(\frac{\delta^2 L}{\delta \lambda^2}\right)$ and $E\left(\frac{\delta^2 L}{\delta \lambda \delta k}\right)$ as well...

Should I find the derivation for $E\left[X^{p}\left( \text{ln}\frac{X_i}{\lambda}\right)^q \right]$ ? Which lead to $\int_0^\infty X^{p}\left( \text{ln}\frac{X_i}{\lambda}\right)^q \frac{k}{\lambda}\left(\frac{x}{\lambda}\right)^{k-1}e^{-\left(\frac{x}{\lambda}\right)^k} $?

Need your help as I have tried my very best to figure it out for weeks... or does it have a better approach for everything...

Answer 1

First, it's easier to compute the information for a single observation, and then multiply the result by $n$ to obtain the information for $n$ IID observations.

That said, the calculation is difficult. The log-likelihood is $$\ell(x \mid k,\lambda) = \log k - k \log \lambda + (k-1) \log x - \frac{x^k}{\lambda^k}. \tag{1}$$ Hence $$\begin{align} \frac{\partial^2 \ell}{\partial \lambda^2} &= \frac{k}{\lambda^2} -k(k+1) \frac{x^k}{\lambda^{k+2}}, \\ \frac{\partial^2 \ell}{\partial \lambda \, \partial k} &= -\frac{1}{\lambda} + \left(1 + k \log \frac{x}{\lambda} \right)\frac{x^k}{\lambda^{k+1}}, \\ \frac{\partial^2 \ell}{\partial k^2} &= - \frac{1}{k^2} - \frac{x^k}{\lambda^k} \log^2 \frac{x}{\lambda}. \end{align} \tag{2}$$

The main challenge, then, is to compute expectations of the form $$\operatorname{E}\left[\frac{X^k}{\lambda^k} \log^m \frac{X}{\lambda}\right] = \int_{x=0}^\infty \frac{x^k}{\lambda^k} \log^m \frac{x}{\lambda} f_X(x \mid k, \lambda) \, dx, \quad m \in \{0, 1, 2\}. \tag{3}$$ To facilitate the calculations, we observe that $\lambda$ is a scale parameter, namely if $X \sim \operatorname{Weibull}(k, \lambda)$, then $Y = X/\lambda \sim \operatorname{Weibull}(k, 1)$. So in general, let $$\begin{align} I(m,k) &= \operatorname{E}\left[\frac{X^k}{\lambda^k} \log^m \frac{X}{\lambda}\right] \\ &= \operatorname{E}[Y^k \log^m Y] \\ &= \int_{y=0}^\infty k y^{2k-1} e^{-y^k} \log^m y \, dy \\ &= k^{-m} \int_{u=0}^\infty u e^{-u} \log^m u \, du. \tag{4} \end{align}$$

The first case ($m = 0$) is easy:

$$I(0,k) = k^0 \int_{u=0}^\infty ue^{-u} \, du = 1. \tag{5}$$

The second case ($m = 1$) is difficult, so we evaluate it without proof:

$$I(1,k) = k^{-1} \int_{u=0}^\infty u e^{-u} \log u \, du = \frac{1 - \gamma}{k}, \tag{6}$$

where $\gamma = \lim_{n \to \infty} H_n - \log n$ is the Euler-Mascheroni constant. The result $(6)$ can be obtained via integration by parts followed by the first formula in the Wikipedia link.

The last case is similarly difficult, so again, we present the result without proof:

$$I(2,k) = \frac{1}{k^2} \left(\frac{\pi^2}{6} - 2 \gamma + \gamma^2\right). \tag{7}$$ This is also obtained through integration by parts followed by one of the definite integral identities in the same Wikipedia link.

With $(5-7)$, we can now proceed to compute $$\operatorname{E}\left[\frac{\partial^2 \ell}{\partial \lambda^2}\right] = \frac{k}{\lambda^2} \frac{k(k+1)}{\lambda^2} I(0,k) = -\frac{k^2}{\lambda^2}, \tag{8}$$ $$\operatorname{E}\left[\frac{\partial^2 \ell}{\partial \lambda \, \partial k} \right] = -\frac{1}{\lambda} + \frac{I(0,k)}{\lambda} + \frac{k}{\lambda} I(1,k) = \frac{1-\gamma}{\lambda}, \tag{9}$$ and $$\operatorname{E}\left[\frac{\partial^2 \ell}{\partial k^2} \right] = -\frac{1}{k^2} - I(2,k) = -\frac{1}{k^2} \left(\frac{\pi^2}{6} + (1 - \gamma)^2 \right). \tag{10}$$ These furnish all of the necessary components to the information matrix.