There are several known bounds for a convex function say $f$ with $L$-Lipschitz continuous gradient, for instance,
\begin{align} \left( x - y \right)^T \left( \nabla f(x) - \nabla f(y)\right) \leq L \| x - y \|_2^2, \forall x, y . \end{align} Exhaustive list can example be found here.
Now, I am wondering if there is any known bound for this case, \begin{align} \left( x - y \right)^T {\color{red} A} \left( \nabla f(x) - \nabla f(y)\right) \leq \ {\color{red}?}, \forall x, y , \end{align} where $A$ is a square matrix .
Questions
- If there is any lower bound, what is it and how to derive that?
- Moreover, what are the requirements on such a matrix $A$? Positive (semi)definite? (except the trivial case where matrix $A$ is a scaled identity)
Attempt: Does the below make sense?
If we invoke Cauchy-Schwartz inequality, then \begin{align} \left( x - y \right)^T {\color{red} A} \left( \nabla f(x) - \nabla f(y)\right) &\leq \underbrace{\left\| {\color{red} A} \right\|_2}_{ = \sigma_{\rm max}\left( {\color{red} A} \right)} \left\| x - y \right\| \ \underbrace{\left\| \nabla f(x) - \nabla f(y) \right\|}_{ \leq L \left\| x - y \right\|} \\ &\leq L \ \sigma_{\rm max}\left( {\color{red} A} \right) \left\| x - y \right\|^2, \quad \forall x, y , \end{align} where $\sigma_{\rm max}\left( {\color{red} A} \right)$ is the maximum singular value of matrix ${\color{red} A} $.