Intuition for derivatives of the log-partition function being moments

42 Views Asked by At

I've been reading Murphy's Advanced Probabilistic ML and the exponential family has popped up as an interesting yet mysterious creature. Many of its properties sound like they should be intuitable, like how the derivatives of the log-partition function correspond to the moments. Formally, given a distribution $$ p(x|\eta)=h(x)\exp(\eta\cdot T(x)-A(\eta)) $$ where $A(\eta)$ is the log-partition function, we have that $\nabla_\eta A(\eta)=\mathbb{E}[T(x)],\nabla^2_\eta A(\eta)=\mathrm{Cov}(T(x))$, and so on. Is there an intuitive, perhaps thermodynamics based explanation for this?