I came accros this very basic lemma :
Let $m$ be an integer, and let $A_1,A_2 \in \mathbb{R}^m$ be two symmetric positive definite matrices. Suppose there exists a constant $C$ such that:
$$ \left( A_1 u,u \right) \leq C \left( A_2u,u\right), \forall u \in \mathbb{R}^m$$
Then, $A_{2}^{-1}A_1$ has real eigenvalues that are bounded by C.
End of lemma.
As it is written, it seems to suggest that the reciprocal is not true. So I wonder if the reciprocal is not true and why it's not true. I'm not really looking for a proof but for understanding. I mean I'm not looking for a proof of the result that does not give me hint on what's going on.
Can anyone give clues on that topic ?
Thks
The question in your title and the question in your text are different.
It is true that $u^T A_1 u \leq C u^T A_2 u$ for all vectors $u$ if and only if $\lambda_{\max}(A_2^{-1}A_1) = C$. This follows from changing variables in the Rayleigh quotient:
\begin{align*} \max_u \frac{u^T A_1 u}{u^T A_2 u} &= \max_{\|A_2^{1/2} u\|=1} u^T A_1 u\\ &= \max_{\|v\|=1}\, v^T \left(A_2^{-1/2} A_1 A_2^{-1/2}\right) v\\ &= \lambda_{\max}\left(A_2^{-1/2} A_1 A_2^{-1/2}\right)\\ &= \lambda_{\max}(A_2^{-1}A_1), \end{align*} where the last equality follows from invariance of the spectrum to cyclic permutations.
That's the answer to the question in your text. For the question in your title: $\lambda_{\max}(A_2^{-1}A_1)$ is not generally the same thing as $\max_{\|u\|=1} u^T (A_2^{-1}A_1) u$, since the matrix in question is not symmetric. It is still true that if $u^T(A_2^{-1}A_1)u \leq C$ for all $\|u\|=1$, then $\lambda_{\max}(A_2^{-1}A_1) \leq C$ (by plugging in the maximum-eigenvalue eigenvector in for $u$) and so $u^T A_1 u \leq C u^T A_2 u$ for all $u$. But I don't believe the converse is true (since $u^T(A_2^{-1}A_1)u$ might exceed $C$ for some non-eigenvector $u$).