How do I relate the definition of a convex function to the geometrical meaning?

Question

I'm having a failure of imagination here and would love some help.

The definition of a convex function (at least as provided in my textbook) is:

A function $f: \Re ^n \rightarrow \Re $ is called convex provided

$f(\tau x + (1- \tau)y) \leq \tau f(x) + (1-\tau)f(y)$

$\forall~ x, y \in \Re ^n $ and each $0 \leq\ \tau \leq 1$

This definition is straightforward enough, but I'm having trouble seeing how exactly it relates to the more familiar, intuitive, geometric definition from elementary calculus. Can anyone help me bridge this gap?

Answer 1

Draw and parametrize a line between points on an upwards opening parabola. Note what values the the function takes when you apply it to the projection of these points onto the x axis.

Note: visualizing a function in n dimensions is tough, so try the graph of a 1 d function.

Answer 2

In elementary (single variable) calculus, convex functions are universally not discussed. To discuss that you would need the concept of gradient, Hessian and convex sets.

What is discussed is the concept of a "concavity". A function can concave up or concave down. $f(x) = x^2$ concave up, and $f(x) = -x^2$ concave down.

In caclcus, you may be aware if $f''(x) > 0, \forall x \in D$, then the function $f$ concaves up over $D$. Similarly, if $f''(x) < 0, \forall x \in D$, then the function $f$ concave down over $D$.

In convex analysis, if $\nabla^2 f(x) \succ 0, \forall x \in D$, meaning that the Hessian is positive definite, then function is strictly convex over domain $D$. If $\nabla^2 f(x) \prec 0, \forall x \in D$, meaning that the Hessian is negative definite, then a function is strictly concave over $D$.

The connection here is that:

1. Hessian is generalized second derivative
2. Positive definite generalizes positivity
3. Negative definite generalizes negativity

On $\mathbb{R}$, $f$ concave up = strictly convex, $f$ concave down = strictly concave.

What about just convex? The intuition from calculus doesn't carry over well because a constant function $f(x) = c$ is convex, but certainly not concaving up or down.