According to the definition, function $f$ is strongly convex if and only if for all $x$ and $y$ where $t>0$,
$$ f(x) \geq f(y) + \langle\nabla f(y), x-y\rangle + \frac{\gamma}{2}\Vert x-y\Vert^2_2 $$
I am trying to understand this using a diagram in the 2-D plane. I am confused about 2 things:
What is the geometrical meaning of $\langle\nabla f(y), x-y\rangle$ in the 2-D plane?
According to my knowledge, $\frac{\gamma}{2}\Vert x-y\Vert^2_2 $ is related to the square of the distance from $x$ to $y$. But I don't understand how this is drawn.
I appreciate your insight on this.
Let's start with ordinary convexity ($\gamma=0$): what is the geometric meaning of $f(x) \geq f(y) + \langle\nabla f(y), x-y\rangle $? The expression on the left is the equation of the tangent plane to the graph of $f$ at the point $x$. The inequality says: the graph of $f$ stays above its tangent plane at every point. That's convexity.
With the quadratic term, the equation $f(y) + \langle\nabla f(y), x-y\rangle + \frac{\gamma}{2}\Vert x-y\Vert^2_2$ describes a paraboloid that is curving upward, with curvature determined by $\gamma$. And the graph of $f$ still has to stay above it. So the condition says: at every point, we can draw a paraboloid of certain curvature which is tangent to the graph of $f$ and lies below it.