Taylor expansion using stirling's approximation

Question

This is from a statistical physics problem, but it is the mathematics behind it that I am stuck on here:

Consider a large number $N$ of distinguishable particles distributed among $M$ boxes.

We know that the total number of possible microstates is $$\Omega=M^N$$ and that the number of microstates with a distribution among the boxes given by the configuration $[n_1, n_2, ..., n_M]$ is given by $$\frac{N!}{\prod_{j=1}^M (n_j)!}\tag{1}$$ In the most likely configuration there are $$n_0 = \frac{N}{M}$$ particles in each box. Let $\Omega_0$ denote the statistical weight of this configuration and $p_0$ its probability. Now consider moving $\delta n$ particles from box $1$ to box $2$, giving the new configuration $$[n_0 − δn, n_0 + δn, n_0, n_0, ..., n_0]$$ Show that $$\fbox{$\color{blue}{\delta\left(\ln\Omega_{\{n\}}\right)\approx -\delta n\ln (n_0)-\frac{(\delta n)^2}{2 n_0}}$}$$ (Hint: you should use Stirling’s approximation here)

The image below shows the situation:

Here is the answer as written by the author:

This might be a bit hard to read so I have typed it out word for word below:

$$\ln\Omega_{\{n\}}=\ln(N!)-\sum_j\ln (n_j!)$$ The change in $\ln\Omega$ due to a change $\delta n$ in ONE box is $$\delta(\ln\Omega_{\{n\}})=-\ln([n_0+\delta_n]!)+\ln(n_0!)$$ Taylor expand $\ln (n!)$ $\color{red}{\text{(To second 2nd order since we are already at a maximum for}}$ $\color{red}{\ln\Omega_{\{n\}}}$$\color{red}{)}$ using Stirling: $$\ln (n!)\approx \underbrace{n_0\ln (n_0)-n_0}_{0th}+\underbrace{\ln (n_0)\cdot\delta_n}_{1st}+\underbrace{\frac{(\delta n)^2}{2n_0}}_{2nd}+\cdots$$ $$\implies \delta(\ln\Omega_{\{n\}})=-\ln (n_0)\cdot\delta n-\frac{(\delta n)^2}{2n_0}$$

The red bracket can be ignored as it was just used in the previous part of the question where we had to show that $$n_0=\frac{N}{M}$$ is the most likely configuration which we did my differentiating and setting to zero to maximize.

---

Finally I get to my question:

I understand everything up to the point where it says "Taylor expand $\ln n!$", firstly where on earth is $n$ even defined? I see $n_0$ and $N$ but not $n$.

I can only assume that the author meant 'Taylor expand $\ln([n_0+\delta_n]!)$' since I fail to see how that expression could ever get the factors $n_0$ and $\delta_n$.

I know that the general Taylor expansion formula is given by $$f(a)+\frac {f'(a)}{1!} (x-a)+ \frac{f''(a)}{2!} (x-a)^2+\frac{f^{(3)}(a)}{3!}(x-a)^3+ \cdots$$

and that Stirlings approximation is as follows $$\ln(k!)\approx k\ln k - k +\frac12\ln k$$

I have used both these formulae, but not both together.

I'm very confused about how to proceed with this, so I naively apply Stirlings approximation first:

$$\ln([n+\delta n]!)\approx (n+\delta n)\ln(n+\delta n)-(n+\delta n)+\frac12\ln(n+\delta n)$$

But now I am completely stuck. Could someone please explain how the author was able to reach the result?

---

EDIT:

A comment below raised an important point. So I will just mention that for the purposes of this question the statistical weight $\Omega$ is equal to the number of microstates.

---

EDIT#2:

I must apologize there was a contradiction in my question due to a typo.

What I needed to show was that $$\fbox{$\color{blue}{\delta\left(\ln\Omega_{\{n\}}\right)\approx -\delta n\ln (n_0)-\frac{(\delta n^2)}{2 n_0}}$}$$ and this is given in blue in the first quotation box, very sorry about this.

---

EDIT#3:

I acknowledge that those of you that answered actually figured out that there must be a similar expression when considering both boxes. So as promised here is part 2 of the question as written by the author:

I'll type it word for word again anyway just in case it's hard to read:

For a change to TWO boxes, moving $\delta n$ from $1$ to $2$, $$\delta\left(\ln\Omega_{\{n\}}\right)=-\ln (n_0)\cdot(-\delta n)-\frac{\left(-\delta n\right)^2}{2n_0}$$ $$\qquad\qquad\qquad\qquad\qquad-\ln (n_0)\cdot(+\delta n)-\frac{\left(+\delta n\right)^2}{2n_0}=-\frac{\delta n^2}{n_0}$$ $$\color{red}{\Huge{\star}}\quad\ln\Omega_{\{n\}}=\ln\Omega_0-\frac{\delta n^2}{n_0}$$

Many thanks.

Answer 1

We assume the Stirling approximation (first order) $$\log(n!)\approx n \log(n) - n = g(n) \tag{1}$$

And the Taylor expansion:

$$g(x)\approx g(x_0)+g'(x_0)(x-x_0)+g''(x_0)\frac{(x-x_0)^2}{2}=\\ =x_0\log(x_0)-x_0 + \log(x_0)\delta+\frac{\delta^2}{2 x_0} \tag{2}$$

where $\delta = x-x_0$ (increment).

Then the change in the box that decrements its counting (box 1) by $\delta$ is

$$\Delta_1(\ln\Omega_{\{n\}})=-\ln((n_0-\delta)!)+\ln(n_0!)=\\= \log(n_0)\delta-\frac{\delta^2}{2 n_0} \tag{3}$$

The change in the other box that increments its counting (box 2) by $\delta$ is

$$\Delta_2(\ln\Omega_{\{n\}})=-\ln((n_0+\delta)!)+\ln(n_0!)=\\ =-\log(n_0)\delta-\frac{\delta^2}{2 n_0} \tag{4}$$

The total change is then $$\Delta(\ln\Omega_{\{n\}})= \Delta_1(\ln\Omega_{\{n\}})+\Delta_2(\ln\Omega_{\{n\}})=-\frac{\delta^2}{n_0} \tag{5}$$

(that's why we need to take a second-order Taylor expansion: because the linear terms cancel - not because "we are at a local maximum" - actually the result is valid for any $n_0$, not only for $n_0=N/M$)

The original statement $$\delta\left(\ln\Omega_{\{n\}}\right)\approx \ln\Omega_0-\frac{\delta n^2}{2 n_0}$$ seems to have two errors: First, if mixes the value of the statistical weight with its change. If we are speaking of the change, then the first "$\delta$" is right, but the initial term $\ln\Omega_0$ is wrong - sanity check: if $\delta n=0$ the change should be zero). Furthermore, the result seems to have a missing two factor (indeed, the copied derivation emphasizes that the first variation is due to one box, but then it seems ot forget about the other one).

Edit: Regarding your edit: your "equation in blue" is only correct if it's understood to be the change in one box (see eq $3$). If it's understood (as it seems) as the total change, then it's wrong. The net change is given by eq. $(5)$

Answer 2

Let us define the following two configurations:

• An "optimal" configuration: $\mathbf n_1 := [n_0, \cdots, n_0]$ .
• The perturbed configuration: $\mathbf n_2 := [n_0 - \delta n, n_0 + \delta n, n_0, \cdots, n_0]$ .

Statistical weights of the two configurations are: $$ \Omega_0 := \Omega_{\{\mathbf n_1\}} = \frac{N!}{(n_0!)^M}\,, \qquad \Omega_{\{\mathbf n_2\}} = \frac{N!}{(n_0-\delta n)!(n_0 + \delta n)! (n_0!)^{M-2}} \,. $$ Taking log of these two weights we get: \begin{align} \ln \Omega_0 =&\; \ln N! - M \ln n_0!\,, \\ \qquad \ln \Omega_{\{\mathbf n_2\}} =&\; \ln N! - \ln (n_0-\delta n)! - \ln(n_0+\delta n)! - (M-2) \ln n_0!\,. \end{align} Log of the number of microstates is proportional to entropy, so let's call these two quantities the entropies of the two configurations (ignoring the proportionality constant, which is called the Boltzmann constant). Change in entropy due to perturbation is: $$\ln \Omega_{\{\mathbf n_2\}} - \ln \Omega_0 = - \ln(n_0-\delta n)! - \ln(n_0 + \delta n)! + 2\ln n_0!\,. \tag{1}$$

Assuming $n_0$ and $n_0-\delta n$ are both fairly large we use Stirling's approximation, nameley $\ln m! \approx m \ln m - m$ for large $m$, to approximate (1): $$ \ln \Omega_{\{\mathbf n_2\}} - \ln \Omega_0 \approx -(n_0 - \delta n) \ln (n_0 - \delta n)-(n_0 + \delta n) \ln (n_0 + \delta n) + 2 n_0 \ln n_0\,. \tag{2}$$ Now we use Taylor expansion to expand $\ln(n_0 \pm \delta n)$ by treating $\delta n$ as small compared to $n_0$. The expansion of $\ln(x+\epsilon)$ around $x$ is given by: $$ \ln(x + \epsilon) = \ln x + \frac{\epsilon}{x} - \frac{\epsilon^2}{2x^2} + \mathcal O\left(\left(\frac{\epsilon}{x}\right)^3\right)\,.$$ Using this in (2), and keeping terms upto quadratic order in $\delta n$ we find: $$ \ln \Omega_{\{\mathbf n_2\}} - \ln \Omega_0 \approx -\frac{(\delta n)^2}{n_0}\,.$$ So the entropy of the perturbed system is given by: $$ \ln \Omega_{\{\mathbf n_2\}} \approx \ln\Omega_0 -\frac{(\delta n)^2}{n_0} \,. \tag{3}$$

The reason why in (3) there is no term linear in $\delta n$ is very physical, an optimal configuration has the highest entropy, therefore change in entropy as you perturb from an optimal configuration must begin at least at the quadratic order.