Consider a function $f:\mathbb{R}^n\to\mathbb{R}$ and denote by $H$ its Hessian matrix. I understand that $H$ provides a measure of the curvature of $f$ in all directions and plays a role in the Taylor approximation of $f$. But I keep encountering the inverse of $H$, and I was wondering if there is some intuition about $H^{-1}$? What does an element $H^{-1}_{ij}$ tell us about about the shape of $f$? Is there a geometric interpretation?
EDIT: Some examples where the inverse Hessian plays a role.
- For instance, in maximum likelihood estimation the negative of the expected value of the Hessian of the log likelihood function is the Fisher information, and the inverse of that quantity gives us the covariance matrix of the estimated parameter.
- In Newton's optimization method, the inverse Hessian of the function tells us how big a step to take toward the optimum.