Here is a related post: Why does Bregman's function require strict convexity?
My question is specifically the second part of the poster's question. I have trouble understanding the wording of "$B_f(x,y)$ is convex in $x$", how can I interpret it mathematically? Because I have only seen that "some function is convex/strictly convex" but never "convex with respect to sth". After that, how do prove that $B_f$ is convex in $x$ but not necessarily in $y$?
Any guidance/answer is deeply appreciated.
Suppose that $f : \mathbb R^n \to \mathbb R$ is convex and differentiable. Recall that the Bregman divergence is given by $B_f(x, y)= f(x) - f(y) - \langle \nabla f(y), x - y \rangle$.
The Bregman divergence is a function $B_f : \mathbb R^n \times \mathbb R^n \to \mathbb R$. A reasonable question to ask about such a function is whether or not it is convex when fixing one argument and varying the other. Let us suppose that $y \in \mathbb R^n$ is fixed, so we are interesting in the mapping $x \mapsto B_f(x, y)$. Re-writing $B_f$ a little, we can see that $B_f(x, y) = (f(x) - \langle \nabla f(y), x\rangle) + (\langle \nabla f(y), y \rangle - f(y))$. The second part is constant in $x$. As a function of $x$, the first part is the sum of a convex function and a linear function, which is convex. Therefore $x \mapsto B_f(x, y)$ is convex for all $y \in \mathbb R^n$.
Things are not so nice when $x$ is fixed. For example, if $f(x) = \exp(x)$, then $B_f(0, y) = 1 - \exp(y) + y \exp(y)$, which is not convex.