Legendre transform optimization problem

112 Views Asked by At

Let $X\sim B(1,p)$ be a Bernoulli random variable wtih $0<p<1$ and $q:=1-p$. Its Legendre transform is

$$L(x):=\sup_{t \in \mathbb{R}} (tx - \ln (q+pe^t))$$

I want to to show that

$$ L(x) = \begin{cases} x \ln(\frac{x}{p}) + (1 - x) \ln(\frac{1 - x}{q}) & x \in [0, 1] \\ \infty & x \notin [0, 1] \end{cases}$$

If $x< 0$ then $tx - \ln (q+pe^t)\to \infty $ as $t \to -\infty$ so the result holds in this case.

If $x>1$ then, since the derivative $x - \frac{pe^t}{q+pe^t}$ of $t\mapsto tx - \ln (q+pe^t)$ is bounded below by $x-1>0$, the mean value theorem tells us that

$$tx - \ln (q+pe^t)\geq t(x-1)\to \infty \hspace{0.2cm} \text{as } t\to\infty$$

so the result holds in this case too.

Am having trouble with the case $x\in [0,1]$. For $x\in (0,1)$ we can set the derivative to zero and obtain the following necessary condition for a maximum:

$$t=\ln \bigg( \frac{xq}{p(1-x)} \bigg)$$

But how can I show that the maximum is indeed attained here?

Thanks a lot for your help.

1

There are 1 best solutions below

6
On BEST ANSWER

Let $f$ denote the function of which the supremum is taken of. Its second derivative is: $$-\frac{qpe^t}{(q + pe^t)^2}$$ That means that $f$ is linear when $q=0$ or $p=0$, concave when $q>0$ and $p>0$, and convex when $q<0$ and $p>0$. You have obtained a maximum only when $f$ is concave.

The remaining cases are $x=0$ and $x=1$. Let me assume that $p$ and $q$ are positive. For $x=0$, the derivative of $f$ is negative, so the supremum is $\lim_{t \to -\infty} f(x)$, whereas for $x=1$ the derivative is positive, so the supremum is $\lim_{t \to \infty} f(x)$.