Fisher's information of normal distribution

Question

Give a sample $X_1,X_2,\ldots,X_n $~$N(\mu,\sigma^2)$ and $\sigma^2=1$. I want to find Fisher's information. I know that: $$ -E(\frac{d^2}{d\mu^2} \ln f(x))=1/\sigma^2. $$ Why this is the Fisher's information in this case? Why the second derivative of normal distribution is not simply equal to zero as expectation of $X$ is $\mu$?