Boltzmann distribution as generalization of Gaussian

Question

I'm trying to gain some intuition on the Boltzmann distribution (also called Gibbs canonical distribution). The definition is very broad: it's an exponential probability distribution that is a function of an energy and temperature, $$ P(x) \propto e^{-\epsilon(x)/kT(x)} $$ where $\epsilon$ and $T$ are the energy and temperature of the state $x$, and $k$ is the Boltzmann constant.

An example is the 1D Gaussian distribution, which has pdf $$P(x) \propto e^{-\frac{(x-\mu)^2}{2\sigma^2}}.$$ Apparently this is the Boltzmann distribution corresponding to the potential energy $$\epsilon(x) = \frac{(x-\mu)^2}{2\sigma^2}.$$

How do I interpret this? I can see that if a point $x$ is far from the mean, then it has high potential energy.

In statistics, the idea of "entropy" is related to the spread of a distribution. Is this saying that a high entropy distribution has high potential energy?

Answer 1

The potential energy ${1\over2}k(x-\mu)^2$ is exactly the well known potential energy of a harmonic spring with offset $\mu$. A spring in a thermal bath where the kinetic energy can be neglected, or can be integrated over will have a position distribution proportional to $e^{-{1\over 2}\beta k (x-\mu)^2}$ where $\beta = {1\over \kappa T}$

Answer 2

Well, one could write pages here. This is a short glimpse from my perspective.
Let $V$ be a nice (convex, smooth, etc.) potential. We know, that the gradient flow \begin{equation} \dot{x}=-\nabla V(x) \end{equation} will relax to the equilibrium, attained at the minimum of $V$. Moreover, $V$ decreases along solutions.
The overdamped Langevin equation is a stochastic pertubation of the gradient flow dynamic above: \begin{equation} dX_{t}=-\nabla V(X_{t})dt+\sqrt{2T}dB_{t} \end{equation} which naturally arises e.g. in statistical mechanics. Classical diffusion theory now tells us that this SDE has an invariant measure of Gibbs-type \begin{equation} \nu_{T}(dx)=\frac{1}{Z_{T}}e^{-\frac{V(x)}{T}}dx \end{equation} where $Z_{T}$ is a normalising constant, often referred to as partition function in statistical mechanics. The Gaussian setting corresponds to a quadratic potential and to an Ornstein-Uhlenbeck process for the SDE. The effect of the random pertubation decreases with a decrease in temperature. On the level of the invariant measure this amounts to a large deviation result for the family $\lbrace\nu_{T}\rbrace_{T>0}$. In particular $\nu_{T}$ weakly converges to the point measure concentrated at the minimiser of $V$. This observation is used in various optimisation algorithms, e.g. simulated annealing.

And since you mentioned entropy: the laws $\mu_{t}$ corresponding to $X_{t}$ evolve according to the Fokker-Planck equation \begin{equation} \partial_{t}\mu=T\Delta\mu-\nabla\cdot(\mu\nabla V). \end{equation} It's an easy exercise to check that the relative entropy $H(\cdot||\mu_{T})$ decreases along solution, i.e. is a Lyapunov function. In fact, since the pioneering work of Felix Otto, we know that this Fokker-Planck equation can itself be interpreted as a gradient flow in an infinite dimensional setting, namely \begin{equation} \dot{\mu}=-\operatorname{grad}_{W}H(\mu||\nu_{T}) \end{equation} where $\operatorname{grad}_{W}$ denotes the formal gradient within Ottos Wasserstein geometry. Since the relative entropy is convex and minimal if $\nu=\mu_{T}$, we recover again the relaxation of the laws towards the invariant Gibbs measure.