Prove that $\lambda _1 ||v||^2 \leq Av \cdot v \leq \lambda _n ||v||^2$.

Question

Given that $A \in \mathbb{M}^{n \times n}(\mathbb{R})$ admits an orthonormal basis of eigenvectors with eigenvalues $\lambda _1 \leq \lambda _2 \leq \dots \leq \lambda _n$. Show that $\lambda _1 ||v||^2 \leq Av \cdot v \leq \lambda _n ||v||^2$ for each $v \in$ $\mathbb{R}^n$.

My suggested (wrong) solution:

We know that: \begin{eqnarray*} Av_i= λ_i e_i \quad \text{ for } \quad i=1,2,…,n \end{eqnarray*}

and

\begin{eqnarray*} v= \sum^n_{i=1} α_i e_i, \end{eqnarray*}

where $\alpha \in \mathbb{R}$ are scalars.

Thus from $Av⋅v$, we get \begin{eqnarray*} Av⋅v &=& A \sum^n_{i=1} α_i e_i ⋅\sum^n_{i=1} α_i e_i\\ &=& \sum^n_{i=1} \lambda_i α_i e_i ⋅\sum^n_{i=1} α_i e_i\\ &=& \sum^n_{i=1} \lambda_i \cdot \left(\sum^n_{i=1} α_i e_i\right)^2\\ &=& \sum^n_{i=1} \lambda_i \cdot \left(\sum^n_{i=1} α_i\right)^2 \cdot \left(\sum^n_{i=1} e_i\right)^2. \end{eqnarray*}

Since the eigenvectors $e_i$ form an orthonormal basis, then $e_i⋅e_i=||e_i||^2=1$, which makes \begin{eqnarray*} \left(\sum^n_{i=1} e_i\right)^2 &=& 1 \\ \Rightarrow Av⋅v &=& \sum^n_{i=1} \lambda_i \cdot \left(\sum^n_{i=1} α_i\right)^2. \end{eqnarray*}

And, \begin{eqnarray*} ||v||^2&=&v⋅v\\ &=& \sum^n_{i=1} \alpha_i e_i \cdot \sum^n_{i=1} \alpha_i e_i\\ &=& \left(\sum^n_{i=1} \alpha_i\right)^2 \cdot \left(\sum^n_{i=1} e_i\right)^2\\ &=& \left(\sum^n_{i=1} \alpha_i\right)^2 . \end{eqnarray*}

So I end up with from $λ_1 ||v||^2\leq Av⋅v\leq λ_n ||v||^2$ to

\begin{eqnarray*} \lambda_1 \left(\sum^n_{i=1} \alpha_i\right)^2 \leq \sum^n_{i=1} \lambda_i \cdot \left(\sum^n_{i=1} α_i\right)^2 \leq \lambda_n \left(\sum^n_{i=1} \alpha_i\right)^2 \end{eqnarray*}

Factoring out the $\left(\Sigma_{i=1}^n \alpha\right)^2,$ \begin{eqnarray*} \lambda_1 \leq \sum^n_{i=1} \lambda_i \leq \lambda_n, \end{eqnarray*}

which is obviously wrong. Can someone please point out where my mistake is?

Answer 1

With

$Ae_i = \lambda_i e_i, \tag 1$

where the

$e_1, e_2, \ldots, e_n \in \Bbb R^n \tag 2$

form an orthonormal basis, we may write any

$0 \ne v \in \Bbb R^n \tag 3$

in the form

$v = \displaystyle \sum_1^n \alpha_i e_i, \tag 4$

whence

$Av = A \left (\displaystyle \sum_1^n \alpha_i e_i \right ) = \displaystyle \sum_1^n \alpha_i A e_i = \sum_1^n \alpha_i \lambda_i e_i, \tag 5$

and

$Av \cdot v = \left ( \displaystyle \sum_1^n \alpha_i \lambda_i e_i \right ) \cdot \left ( \displaystyle \sum_1^n \alpha_i e_i \right )$ $= \displaystyle \sum_{i,j = 1}^n \lambda_i \alpha_i \alpha_j e_i \cdot e_j = \sum_{i,j = 1}^n \lambda_i \alpha_i \alpha_j \delta_{ij}, \tag 6$

since the orthonormality of the $e_i$ may be written

$e_i \cdot e_j = \delta_{ij}; \tag 7$

then (6) becomes

$Av \cdot v = \displaystyle \sum_1^n \lambda_i \alpha_i^2 \ge \sum_1^n \lambda_1 \alpha_i^2 = \lambda_1 \sum_1^n \alpha_i^2 = \lambda_1 v \cdot v =\lambda_1 \Vert v \Vert^2, \tag 8$

since (4) yields

$\Vert v \Vert^2 = \displaystyle \sum_1^n \alpha_i^2, \tag 9$

as the reader may easily verify. In a manner similar to (8) we also have

$Av \cdot v = \displaystyle \sum_1^n \lambda_i \alpha_i^2 \le \sum_1^n \lambda_n \alpha_i^2 = \lambda_n \sum_1^n \alpha_i^2 = \lambda_n v \cdot v =\lambda_n \Vert v \Vert^2; \tag{10}$

combining (8) and (10) we obtain

$\lambda_1 \Vert v \Vert^2 \le Av \cdot v \le \lambda_ n \Vert v \Vert^2, \tag{11}$

the sought-for result.

Answer 2

Your error comes from the fact that in general :

$$\left(\sum^n_{i=1} α_i e_i\right) \ne \left(\sum^n_{i=1} α_i\right) \cdot \left(\sum^n_{i=1} e_i\right)$$

Besides, the inequalities you want to prove are a property of "Rayleigh quotient"

$$\lambda_{min} \leq R(v):=\dfrac{Av.v}{\|v\|^2} \leq \lambda_{max}$$

(https://en.wikipedia.org/wiki/Rayleigh_quotient) which can be proven by an elementary and elegant barycentric argumentation.

Answer 3

You made a mistake here $$ \sum^n_{i=1} \lambda_i α_i e_i ⋅\sum^n_{i=1} α_i e_i= \sum^n_{i=1} \lambda_i \cdot \left(\sum^n_{i=1} α_i e_i\right)^2.$$ To fix it, one has to change $\sum^n_{i=1} α_i e_i=\sum^n_{j=1} α_j e_j$ and $e_i\cdot e_j=\delta_{ij}$. In fact \begin{eqnarray*} Av⋅v &=& A \sum^n_{i=1} α_i e_i ⋅\sum^n_{i=1} α_i e_i\\ &=& \sum^n_{i=1} \lambda_i α_i e_i ⋅\sum^n_{j=1} α_j e_j\\ &=& \sum^n_{i,j=1} \lambda_iα_iα_j( e_i\cdot e_j)\\ &=& \sum^n_{i,j=1} \lambda_i α_iα_j\delta_{ij}\\ &=&\sum^n_{i,j=1} \lambda_i(α_i)^2. \end{eqnarray*} Since $\lambda_1\le\lambda_i\le\lambda_n$, one has $$ \sum^n_{i,j=1} \lambda_1(α_i)^2\le Av\cdot v\le \sum^n_{i,j=1} \lambda_n(α_i)^2 $$ or $$ \lambda_1\|v\|^2\le Av\cdot v\le \lambda_n\|v\|^2. $$