Legendre transform of a scalar function - getting stuck at inverting vectors

Question

I'm interested in finding the Legendre transform of the following function $$ f(x) = - \log(\langle a,x\rangle) $$ where $a$ and $x$ are both $n$-dimensional (column) vectors. The values of $x$ and $a$ are such that the inner product is always positive

I wish to find the Legendre transform of $f(x)$. My attempt is below $$ \langle p,x\rangle - f(x) = \langle p,x\rangle + \log(\langle a,x\rangle) $$ The gradient is given as $$ \frac{d}{dx} = p + \frac{a}{\langle a,x\rangle} $$ And the hessian is $$ \frac{d^2}{dxdx^{T}} = -\frac{aa^{T}}{\langle a,x\rangle^2} $$ (I think). This hessian is negative, which means the fixed point we find is a maximum - all good so far.

From here I would be like to be able to write something like the following $$ f^{\ast}(p) = -\log(\langle a,x^{\ast}\rangle) $$ where $x^{\ast} = g(p)$, which I find by inverting that first derivative (when it is at a minimum). However, I don't know how to do this. $$ \frac{a}{p} = a^{T}x $$ The thinking here is to somehow invert $a$, but in general one can't invert vectors in matrix multiplication, since inverses only exist for square matrices. What, then, should I do to attack this problem?

The other classic way to solve this problems is to invert the gradient of the function, i.e., to find the following function $$ (\frac{d}{dx}f)^{-1}(p) $$ but since this gradient ($\nabla f$) is again a vector (of partial derivatives), I am again stuck on how to proceed

Answer 1

Assume the $a \neq 0$. The domain of $f$ is $\mathcal D = \{x : \langle a, x\rangle > 0\}$ and the Legendre transform is $f^*(p) = \sup_{x \in \mathcal D} \langle p, x \rangle + \log \langle a, x\rangle$.

Suppose it is not true that $p = - \alpha a$ for some $\alpha \geq 0$, then there exists an $x$ with $\langle a, x \rangle > 0$ and $\langle p, x \rangle > 0$. By increasing the magnitude of this $x$ arbitrarily you see that $f^*(p) = \infty$.

On the other hand, if $p = -\alpha a$ for $\alpha \geq 0$, then $$ f^*(-\alpha a) = \sup_{x \in \mathcal D} -\alpha \langle a, x \rangle + \log \langle a, x\rangle = \log(1/\alpha) - 1 $$ The reason you encountered trouble is because $\nabla f(\mathbb R^n) \neq \mathbb R^n$. In fact, it lives in a subspace. Thinking about the geometric definition of the Legendre transform may be helpful here.

Answer 2

We have $a \ne 0$ since we need $\langle a, x \rangle \ne 0$.

$$f^*(p)=\sup_{x: (\langle x, a\rangle > 0) } (\langle p, x \rangle +\log \langle a, x \rangle)$$

Differentiating, we have $$\nabla_x(f)= p + \frac{a}{\langle a, x \rangle}=0$$

$$ p=-\frac{a}{\langle a, x \rangle}$$

• If $p$ is not a negative multiple of $a$, let $x = k(|p|a+|a|p)$ where $k>0$ is a constant to be determined:

$$\langle p, x \rangle = k(|p|\langle p, a\rangle + |a| |p|^2)=k|p|(\langle p, a\rangle + |a||p|)$$ which is nonnegative by Cauchy-Schwarz inequaltiy.

Similarly, $$\langle a, x \rangle = k(\langle a, |p| a + |a| p\rangle=k(|p||a|^2 + |a|\langle a, p \rangle)=k|a|(|p||a| + \langle a,p \rangle ) $$

which is positive by Cauchy-Schwarz inequality. We can choose $k$ to be arbitrarily large to make $\langle a, x \rangle$ to exceed $1$ and hence $\langle p, x \rangle, + \log \langle a, x \rangle$ can be chosen to go to $\infty$.

• If $p$ is a negative multiple of $a$, $p =-ka$, $k>0$, $k = \frac1{\langle a, x \rangle}$, $\langle a, x \rangle =\frac1k$ and $\langle p, x\rangle=\langle -ka, x\rangle =-1.$ That is $f^*(-ka) = -1 - \log k$.