Maximizing multinomial distribution with constraints using Lagrange multipliers

260 Views Asked by At

Use methods of Lagrange multipliers to maximize $$W (N_{1}, \dots, N_{M}) = \frac{N!}{\prod_{j=1}^{M} N_{j}!}$$ under the constraints $\sum N_{j} = N =$ constant, $\sum E_{j} N_{j} = \mathcal{E} =$ constant. Assume $N_{j}$'s are continuous and $N$ large enough to use Stirling's approximation.

After turning the problem into the maximization of $$\ln W = N \ln N - N - \sum (N_{j} \ln N_{j} - N_{j}),$$ I proceeded to setting up equations with Lagrange multipliers $\lambda_{1}, \lambda_{2}$ $$ \frac{\partial}{\partial N_{i}} \left[{\ln W - \lambda_{1} ({\sum N_{j} - N}) - \lambda_{2} ({\sum E_{j} N_{j} - \mathcal{E}}}) \right] = 0 . $$

However, I encountered a problem when calculating the partial derivative of $\ln W$: \begin{align*} \frac{\partial}{\partial N_{i}} \ln W &= \frac{\partial}{\partial N_{i}} \left[ N \ln N - N - \sum ({ N_{j} \ln N_{j} - N_{j}}) \right] \\ &= \frac{\partial}{\partial N_{i}} ({N \ln N - N}) - \frac{\partial}{\partial N_{i}} \sum ({ N_{j} \ln N_{j} - N_{j}}) & (N \textrm{ constant}) \\ &= 0 - ({\ln N_{i} + N_{i} \cdot \frac{1}{N_{i}} - 1}) \\ &= - \ln N_{i} \end{align*} Here, I imposed the constraint that $N$ is constant. However, it occurred to me that, depending on when the constraint is used, there can be different results: \begin{align*} \frac{\partial}{\partial N_{i}} \ln W &= \frac{\partial}{\partial N_{i}} \left[N \ln N - N - {\sum ( N_{j} \ln N_{j} - N_{j}) }\right] \\ &= \frac{\partial}{\partial N_{i}} [N \ln N - N - \sum N_{j} \ln N_{j} + \sum N_{j}] \\ &= \frac{\partial}{\partial N_{i}} [N \ln N - N - \sum N_{j} \ln N_{j} + N] \\ &= \frac{\partial}{\partial N_{i}} (N \ln N) - \frac{\partial}{\partial N} \sum N_{j} \ln N_{j} \\ &= 0 - ({\ln N_{j} + N_{j} \cdot \frac{1}{N_{j}}}) \\ &= - \ln N_{j} - 1 \end{align*} or (treating $N = \sum N_{j}$ as a function of $N_{j}$'s) \begin{align*} \frac{\partial}{\partial N_{i}} \ln W &= \frac{\partial}{\partial N_{i}} \left[N \ln N - N - {\sum N_{j} (\ln N_{j} - N_{j}) }\right] \\ &= \frac{\partial}{\partial N_{i}} ({N \ln N - N}) - \frac{\partial}{\partial N_{i}} {\sum ( N_{j} \ln N_{j} - N_{j}}) \\ &= {\frac{\partial}{\partial N_{i}} ({\sum N_{j} \ln \sum N_{j} - \sum N_{j}}}) - ({\ln N_{j} + N_{j} \cdot \frac{1}{N_{j}} - 1}) \\ &= \ln \sum N_{j} + \sum N_{i} \cdot \frac{1}{\sum N_{j}} - 1 - ({\ln N_{j} + N_{j} \cdot \frac{1}{N_{j}} - 1}) \\ &= \ln \sum N_{j} - \ln N_{j} \end{align*} When should I use the constraint $N = \sum N_{j} =$ constant?

1

There are 1 best solutions below

4
On

Maximization actually does come from the expression :

$$ d(\ln(W))=0 $$ Giving

Lagrange multiplier on :

$$ \mathcal{L}(W,\lambda_1,\lambda_2)=\ln(W)-\lambda_1(\sum N_j-N)-\lambda_2(\sum e_iN_j-\epsilon)$$

$$d\mathcal{L}=0$$

Factorize by $dN_i$ in the sum, the factors (coordinates of the $dN_i$ in the sum is null because $dN_i$ is a differential)

You will get the result.