Inequality for smooth convex function defined in non Euclidean norm

735 Views Asked by At

Let $\mathcal{X} $ be a closed, convex and bounded set. Consider a convex function $f : \mathcal{X} \rightarrow \mathbb{R}$ that is differentiable with $L$ Lipschitz smooth gradient w.r.t a norm $\| \cdot \|$ that is not necessarily the Euclidean norm. Let $\| \cdot \|_*$ be the dual norm of $\|\cdot \|$, i.e., $\|x\|_* = \max \{ \langle x, u \rangle \mid \|u \| = 1\}$. For every $x, y \in \mathcal{X}$, does the following inequality hold?

$$ f(x) - f(y) - \langle \nabla f(y), x - y \rangle \ge \frac{1}{2 L} \| \nabla f(x) - \nabla f(y) \|^2_* . $$

The above is true when $\|\cdot\|$ and $\|\cdot\|_*$ are both the Euclidean norm which can be found in many textbooks, however, I can't find any statement regarding the non-Euclidean case. The proof used for the Euclidean case doesn't seem to apply to the more general case.

1

There are 1 best solutions below

0
On BEST ANSWER

After some research, I think the answer is yes. Since LHS is the Bregman divergence of $f$, let $f^*$ be the convex conjugate of $f$, we have $$ D_f(x,y) = D_f^*(\nabla f(y), \nabla f(x)) \ge \frac{1}{2L} \| \nabla f(y) - \nabla f(x) \|_*^2, $$ where the first equality is due to the duality property of Bregman divergence (see https://en.wikipedia.org/wiki/Bregman_divergence), and the second inequality is due to the fact that $f$ is $L$-Lipschitz smooth if and only if $f^*$ is $\frac{1}{L}$-strongly convex. (see Theorem 1 of http://ttic.uchicago.edu/~shai/papers/KakadeShalevTewari09.pdf).