Trying to show $\Vert A (\sum_{j=1}^n v_j)\Vert_2^2 = \sum_{j=1}^n\lambda_j^2$?

Question

I have been reading through some lecture notes and I think the author must be mistaken in a particular statement he makes.

Suppose $A$ is a symmetric positive definite matrix. Then there exists an orthonormal basis $v_1, \dots, v_n \in \mathbb{R}^n$ of eigenvectors with eigenvalues $0 < \lambda_1 \le \dots \le \lambda_n < \infty$. As the eigenvectors are orthonormal this means that we can write:

$$\Vert A (\sum_{j=1}^n v_j)\Vert_2^2 = \sum_{j=1}^n\lambda_j^2$$

I don't think this is correct? I have

$$\Vert A (\sum_{j=1}^n v_j)\Vert_2^2 = \Vert \sum_{j=1}^n \lambda_j v_j \Vert_2^2.$$ Taking the $2$ norm gives

$$\sum_{i=1}^n |\sum_{j=1}^n \lambda_j v_{j_i}|^2,$$

and I don't think this can be reduced further?

Answer 1

Your lecture notes are correct. Begin by orthogonally diagonalizing $A$ as $A = QDQ^t$ where $Q$ is the orthogonal matrix whose columns are the orthonormal eigenvectors of $A$ and $D$ is the diagonal matrix whose entries are the eigenvalues of $A$. Note that since $A$ is symmetric, $(QDQ^t)^t = Q^tDQ$. Put $v = \sum_i v_i$. If we expand $\Vert A\sum_i v_i \Vert^2$ with the dot product, we have

$$ \Vert A v\Vert^2 = A v \cdot Av = v\cdot A^tAv = v^tA^tAv = v^tA^2v = v^tQ^tD^2Qv. $$ Writing out this right-hand side: $$ v^tQ^tD^2Qv = v^t\begin{pmatrix} & v_1 & \\ & \vdots & \\ & v_n & \end{pmatrix} D^2\begin{pmatrix} & & \\ v_1 & \dotsb & v_n \\ & & \end{pmatrix}v= \sum_{i,j}\lambda_i^2v_i^tv_j = \sum_{i,j}\lambda_i^2\delta_{i,j} = \sum_i\lambda_i^2 $$ since $v_i^tv_j = \delta_{i,j}$ for an orthonormal list $v_1, \ldots, v_n$.

Answer 2

The author is correct. Note that $v_{i}^{T}v_{j}=0$ for $i \neq j$, and that $v_{i}^{T}v_{i}=1$ for all $i$. When you expand the sum you'll find that most of the terms vanish and the remaining terms simplify to give you the desired result.