Notation and proof of statement about convex functions

28 Views Asked by At

I am reading my course notes on convex optimisation, but the notes provide a notation that I am unfamiliar with. The notes makes the claim that if $f:\mathbb{R}^n\mapsto\mathbb{R}$ is convex, ${\bf x}$, ${\bf y}\in\mathcal{D}(f)$, ${\bf a}\in\partial f({\bf x})$ and ${\bf b}\in\partial f({\bf y})$, then $({\bf a}-{\bf b})^\mathrm{T}({\bf x}-{\bf y})\geq{\bf 0}$. I therefore would like to check the following.

  1. $\mathcal{D}$ either represents domain or preimage, which is it? I suspect it is domain because of the it starts with 'D', but nevertheless it is worthwhile to check.

  2. What exactly are $\partial f({\bf x})$ and $\partial f({\bf y})$? In physics, I know that $\partial V$ represents the surface bounding a volume $V$, are they related? If it is referring to a "surface", what is the proper mathematical way to define $\partial f({\bf x})$ and $\partial f({\bf y})$?

  3. Assuming I have gotten the interpretations of these right, I wanted to prove it as follows: using without proof a claim that further appears in the notes, I have $f({\bf x}+{\bf d})-f({\bf x})\geq{\bf a}^\mathrm{T}{\bf d}$ and $f({\bf y}+{\bf d})-f({\bf y})\geq{\bf b}^\mathrm{T}{\bf d}$. If I could "subtract" these two inequalities intuitively I know I will be close, but I just can't seem to figure out how. What have I done wrong here? 

1

There are 1 best solutions below

0
On BEST ANSWER

Here $\mathcal{D}(f)$ should be the domain of $f$: $$ \mathcal{D}(f) := \{ x \in \mathbb{R}^n \mid f(x) < \infty \}. $$

Moreover, $\partial f(x)$ is the subdifferential of $f$ at $x$. You can think of it as a generalization of the gradient. In particular, you always have the inequality

$$ f(y) \geq f(x) + \zeta^{\mathsf{T}}(y - x), \; \forall x, y \in \mathcal{D}(f), \; \forall \zeta \in \partial f(x). $$

Your idea for proving the inequality is correct. Applying the above inequality twice, with the order of $y$ and $x$ reversed, you have

$$ f(y) - f(x) \geq \zeta^{\mathsf{T}}(y - x), \; \forall \zeta \in \partial f(x). \\ f(x) - f(y) \geq w^{\mathsf{T}}(x - y), \; \forall w \in \partial f(y). $$

Add them together and the LHS cancels; rearranging will give you

$$ (\zeta - w)^{\mathsf{T}}(x - y) \geq 0, $$

and since the choice of $x$ and $y$ was arbitrary the inequality follows. (Note that this is exactly the proof you suggested, with $d = (y - x)$ in the first invocation and $d = (x - y)$ in the second one.)