What lemma for product of derivatives equals the n-derivative?

Question

Why must one require regularity in order for Fisher information to be $E\bigg( \frac{\partial^2 l(\theta, X)}{\partial \theta^2} \bigg)$?

Rather than

$E\bigg( \frac{\partial l(\theta, X)}{\partial \theta}\frac{\partial l(\theta, X)}{\partial \theta}^T \bigg)$

Since my notes say that "under sufficient regularity conditions", then:

$I(\theta)=E\bigg( \frac{\partial^2 l(\theta, X)}{\partial \theta^2} \bigg)$

However, what's the lemma that says that the product equals 2nd derivative?

Answer 1

My impression is that you got something wrong here. Note that $$\frac{\partial^2 l(\theta, X)}{\partial \theta^2} $$ is a second derivative, while $$\frac{\partial l(\theta, X)}{\partial \theta}\frac{\partial l(\theta, X)}{\partial \theta}^T$$ is a product of two first-order derivatives. As such, they are surely not equal in general: consider for example the function $$l(\theta,X) = \theta \cdot X.$$

Try reading the wikipedia page on the Fisher information and compare it with your notes.