Modify maximum entropy problem to fix specific Lagrange multiplier

40 Views Asked by At

I have an optimization problem of the form

$\min_p -H(p) \text{ s.t. } \sum_{x \in \mathcal{X}} p(x)=1 \text{ and } g_i(p)=c_i \text{ for } i=1, \dots, m$

where $p \in [0,1]^{|\mathcal{X}|} $ is a discrete probability distribution over a set $\mathcal{X}$.

From e.g. Chapter 3 of Wainwright/Jordan 2008, I know that the solution to the above optimization problem is of the form

$p(x) = \exp\left(\lambda_0 + \sum_{i=1}^n \lambda_i g_i(x) \right)$.

for some constants $\lambda_i \in \mathbb{R}$ (the Lagrange multipliers).

In the solution $p(x)$ above I want one of the coefficients $\lambda_i$ to have a fixed value, say for example I want to fix $\lambda_1=5$. Is there a way to enforce a constraint like $\lambda_1=5$ in the original maximum entropy problem?