The eigenvalue bounds for an integration of Hessian matrices along a line segment given a twice differentiable multivariate function

18 Views Asked by At

In the paper [1], I encounter the following statement:

${(1)~ m \mathbf{I} \preccurlyeq \nabla^{2} \phi(\mathbf{x} + t \mathbf{s}) \preccurlyeq M \mathbf{I} }$

${~~~~~~}$for $\forall t \in [0, 1]$, ${~0 < m \leq M}$, ${~}$and fixed ${~\mathbf{x}, \mathbf{s} \in \mathbb{R}^{d} }$

${\Rightarrow ~(2)~ m \mathbf{I} \preccurlyeq \int_{0}^{1} \nabla^{2} \phi(\mathbf{x} + t \mathbf{s})~ d t \preccurlyeq M \mathbf{I}, }$

where the matrix inequality ${ \mathbf{B} \preccurlyeq \mathbf{A} }$ with ${\mathbf{A}, \mathbf{B} \in \mathbb{R}^{d \times d}}$ denotes that the subtraction between two matrices ${\mathbf{A} - \mathbf{B}}$ results in a positive semidefinite matrix: ${ \mathbf{0} \preccurlyeq \mathbf{A} - \mathbf{B} }$, the notation ${\nabla^{2} \phi(\mathbf{x})}$ denotes the Hessian matrix of a twice differentiable function ${\phi: \mathbb{R}^{d} \rightarrow \mathbb{R}}$ at ${\mathbf{x} \in \mathbb{R}^{d}}$, the constants range ${0 < m \leq M}$, and the matrix ${\mathbf{I} \in \mathbb{R}^{d \times d}}$ denotes the identity matrix.

The result ${(1) \Rightarrow (2)}$ is counter-intuitive because ${\int_{0}^{1} \nabla^{2} \phi(\mathbf{x} + t \mathbf{s}) ~d t}$ describes a sum of positive definite matrices, from which the eigenvalues keep the same range ${[m, M]}$ after the sum. I try to understand this statement. I write down below what I tried, and I will appreciate your comments.

Firstly, I list the boundness for the Hessian's eigenvalues ${(1)}$ in the range of all ${t \in [0, 1]}$ as follows:

${(3)~ m \mathbf{I} \preccurlyeq \nabla^{2} \phi(\mathbf{x}) \preccurlyeq M \mathbf{I}~}$ for ${~t = 0}$

${\vdots}$

${m \mathbf{I} \preccurlyeq \nabla^{2} \phi(\mathbf{x} + t \mathbf{s}) \preccurlyeq M \mathbf{I}~}$ for ${~t \in (0, 1)}$

${\vdots}$

${(4)~ m \mathbf{I} \preccurlyeq \nabla^{2} \phi(\mathbf{x} + \mathbf{s}) \preccurlyeq M \mathbf{I}~}$ for ${~t = 1}$,

and then I am going to sum up all the above inequalities (3-4), assuming that the matrix inequalities are preserved due to the positivity of the constants ${0 < m \leq M}$. Since we are integrating with respect to ${t \in [0, 1]}$, we have the length for the integration ${\int_{0}^{1} 1 ~dt}$, and thus, the sum of the inequalities (3-4) reads

${(5)~ m \mathbf{I} \int_{0}^{1} 1~dt \preccurlyeq \int_{0}^{1} \nabla^{2} \phi(\mathbf{x} + t \mathbf{s}) ~dt \preccurlyeq M \mathbf{I} \int_{0}^{1} 1~dt}$.

Because the length for the integration ${\int_{0}^{1} 1 ~dt}$ is positive, we can also divide ${(5)}$ and achieve

${(6)~ m \mathbf{I} \frac{\int_{0}^{1} 1~dt}{\int_{0}^{1} 1~dt} \preccurlyeq \frac{1}{\int_{0}^{1} 1~dt} \int_{0}^{1} \nabla^{2} \phi(\mathbf{x} + t \mathbf{s}) ~dt \preccurlyeq M \mathbf{I} \frac{\int_{0}^{1} 1~dt}{\int_{0}^{1} 1~dt}}$,

which is the same as the desired result ${(2)}$.

If the integration range differs as ${t \in [0, 2]}$ for the same fixed ${\mathbf{x}}$ and ${\mathbf{s}}$, then we instead have

${(7)~ m \mathbf{I}\frac{\int_{0}^{2} 1~dt}{\int_{0}^{2} 1~dt} \preccurlyeq \frac{1}{\int_{0}^{2} 1~dt} \int_{0}^{2} \nabla^{2} \phi(\mathbf{x} + t \mathbf{s}) ~dt \preccurlyeq M \mathbf{I} \frac{\int_{0}^{2} 1~dt}{\int_{0}^{2} 1~dt}}$

${\leftrightarrow~ m \mathbf{I} \preccurlyeq \frac{1}{2} \int_{0}^{2} \nabla^{2} \phi(\mathbf{x} + t \mathbf{s}) ~dt \preccurlyeq M \mathbf{I} ~(8) }$

Is my understanding correct? Also, how should I argue for the used assumption that the matrix inequalities are preserved due to the positivity of the constants?

Thanks a lot in advance.

[1] Y. Xie, R. H. Byrd, J. Nocedal, Analysis of the BFGS method with errors, SIAM Journal on Optimization 30 (1) (2020) 182–209. doi: 10.1137/19M1240794