Meaning of $LI \succeq \nabla^2 f(x) \succeq \mu I$ in optimization

Question

In these notes, it is said that a function is convex iff

$$\nabla^2 f(x) \succeq 0$$

In many literatures $\succeq$ is taken as element-wise greater or equal to. But in this case, $\succeq$ means positive semidefinite. Okay.

Now, on page 41 it says:

$$LI \succeq \nabla^2 f(x) \succeq \mu I$$

It seems that $\succeq$ has been overloaded without clarification.

Can someone clarifies what this means? Does this mean that the diagonal terms on the Hessian is element-wise greater than $\mu$ or less than $L$? What does it have to do with the eigenvalues of hessian?

Answer 1

If $A \succeq 0$ means positive semidefinitivity of $A$, one denotes by $A \succeq B$ the fact that $A - B \succeq 0$, or equivalently, that $$ \def\<#1>{\left<#1\right>}\<Ax,x> \ge \<Bx,x> \qquad x \in X $$ ($X$ is the space on which the operators $A$, $B$ act). Hence $LI \succeq f''(x) \succeq \mu I$ means that $$ L\def\n#1{\left\|#1\right\|}\n u^2 \ge \<f''(x)u, u> \ge \mu \n u^2 , \qquad u \in X $$ In terms of spectral values, this holds exactly iff $\sigma\bigl(f''(x)\bigr) \subseteq [\mu, L]$, that is, all spectral values of the Hessian lie between $\mu$ and $L$.