LogLiklihood Function and Fisher Information Martix

104 Views Asked by At

Given PDF: $$ f(x) = \frac{3\theta^3}{(x_i +\theta)^4} $$ $$ x \in [0, \infty]$$

I need to find the Rao-Cramer lower bound, but I am having trouble getting started. Here is what I have so far, any help would be greatly appreciated:

$$L(x;\theta) = \prod_{i=1}^n f(x_i;\theta) = \prod_{i=1}^n \frac{\theta^3} {(x_i +\theta)^4}$$

Taking Log of both sides: $$ l = log(L) = log[\prod_{i=1}^n \frac{\theta^3} {(x_i+\theta)^4}] $$

Then converting to a summation: $$ log(L) = \sum_{i=1} ^n log\frac{\theta^3} {(x_i+\theta)^4} $$

Rearranging: $$ log(L) = \sum_{i=1} ^n 3log(\theta) - \sum_{i=1}^n {4log(x_i +\theta)} $$

$$ = 3log(\theta) - 4\sum_{i=1}^n {log(x_i +\theta)} $$

$$ \frac{\partial{l}}{\partial{\theta}} = \frac{3}{\theta} - 4\sum \frac{1}{(x_i +\theta)}$$

At this point, I am not quite sure how to proceed.

1

There are 1 best solutions below

2
On BEST ANSWER

So $f_X(x) = \dfrac{3 \theta^3}{(\theta+x)^4} \mathbb{1}_{[0,\infty)}(x)$. (It's good habit to include the bounds for $x$ with pdf).

I'm assuming $X_1,\dots,X_n$ are a random sample from this distribution, and that you're looking for the CRLB of all unbiased estimators of $\theta$. The CRLB is given by $\dfrac{1}{I_n(\theta)}$, where $$I_n(\theta) = E \left[ \left( \dfrac{ \partial \ell(\theta)}{\partial \theta}\right)^2 \right] = - E \left[ \dfrac{\partial^2}{\partial \theta^2} \ell (\theta) \right] $$ (https://en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93Rao_bound#Scalar_unbiased_case)

Often times, it's easier to use the rightmost formula and do two partial derivatives instead of squaring the partial derivative of the log likelihood.

$$ L(\theta) = \theta^{3n} \prod_{i=1}^n (x_i + \theta)^{-4} \implies \ell (\theta) = 3n \ln(\theta) + \sum_{i=1}^n -4\ln(x_i+\theta) $$ Then we calculate $$ \frac{\partial \ell}{\partial \theta} = \frac{3n}{\theta} + \sum_{i=1}^n \frac{-4}{x_i+\theta} $$ and $$ \frac{\partial^2 \ell}{\partial \theta^2} = - \frac{3n}{\theta^2} + \sum_{i=1}^n \frac{4}{(x_i+\theta)^2} $$

Then, the Fisher Information $I_n(\theta)$ equals $\frac{3n}{\theta^2} - n E \left[ \frac{4}{(x+\theta)^2} \right]$, where the last term comes from the fact that our variables are iid. So you just need to take the expectation . . . should get that it's $\frac{12}{5 \theta^2}$.

Finally, the CRLB is $1/I_n(\theta) = 1/ \left( \frac{3n}{\theta^2} - \frac{12n}{5\theta^2} \right) = \frac{5 \theta^2}{3n}.$