Set $P(x)$ as probability distribution function, which satisfies(constraints)
- $x\in[0,1]$
- $P(x)\ge 0$, $\int^1_0P(x)dx=1$
- $P(x)\rightarrow x^2$ while $x\rightarrow 0$
- $P(x)=P(1-x)$
Then, how to find the maximum entropy distribution of it? In other words, does there exist a form of smooth function $P(x)$ such that $\Lambda(x):=-\int^{1}_0 P(x)\ln P(x)dx$ is the maximum value, and what is it?
One came up with the idea when meeting an other question related with it, where $P(x)$ satisfies
- $x\in S$
- $\int_S f_i(x)P(x)dx=\alpha_i$
The maximum entropy distribution with constraints is $$P(x)=\exp \left(\sum_{j=0}^{n} \lambda_{j} f_{j}(x)\right)$$ where $\lambda_j$ can be calculated with the constraints.
But how to solve my problem especially the $3\text{rd}$ condition? And the $4\text{th}$ condition may be useless as we can transform the interval to $[0,1/2]$ by symmetry.
Hopefully someone can help me figure this out. Thanks!
No, there does not exist such a maximum entropy distribution.
As you note, the fourth condition just reduces to studying probability distributions on $\left[0,\frac{1}{2}\right]$; for simplicity, I will just ignore it.
The maximum entropy distribution on $[0,1]$ is well-known to be the uniform, which has entropy $0$. But now consider the distribution $$P_\epsilon(x)=\begin{cases} \frac{1-\frac{\epsilon^3}{3}}{1-\epsilon} & \epsilon\leq x\leq1 \\ x^2 & 0\leq x\leq\epsilon \end{cases}$$ This has entropy \begin{align*} -\int_0^1{P_\epsilon(x)\ln{\!(P_\epsilon(x))}\,dx}&=-\int_0^{\epsilon}{2x^2\ln{(x)}\,dx}-\left(1-\frac{\epsilon^3}{3}\right)\ln{\!\left(\frac{1-\frac{\epsilon^3}{3}}{1-\epsilon}\right)} \\ &=-\epsilon^2(2\ln{\!(\epsilon)}-1)-\left(1-\frac{\epsilon^3}{3}\right)\ln{\!\left(\frac{1-\frac{\epsilon^3}{3}}{1-\epsilon}\right)} \\ &=-\epsilon+\epsilon^2\left(\frac{1}{2}-2\ln{\!(\epsilon)}\right)+o(\epsilon^2) \\ &\to0 \end{align*} as $\epsilon\to0^+$. Thus the supremum of entropy over distributions satisfying (1-3) is $0$, which we already know is attained only for the uniform distribution. And the uniform distribution fails to satisfy (3).